asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ildar Absalyamov <>
Subject Homogeneous lists with nullable items
Date Wed, 16 Dec 2015 07:12:34 GMT
Hi devs,

Recently I have been playing around with lists and functions, which receive/return list parameters/values.
I have noticed one particular issue, which seems to be incorrect.
As you might know internally we do support 2 types of lists homogeneous, where all the items
are untagged and the item type is stored in type definition, and heterogeneous, where items
on contrary are tagged, and the list item type is effectively ANY.
The decision which of two types would be used is usually done by parser or is altered by IntroduceEnforcedListTypeRule,
which effectively turns heterogenous list into homogenous if all the items have the same type.
Right now only we allow homogeneous lists to be defined as a field in some type, we also restrict
the item type to be only non-nullable type:
create type listType {
  “list”:[int64]   // [int64?] is not possible

This constraint spans both of the language level as well as serialization. Under that restriction
the only way to load the list, which contains null values, would be to make the appropriate
field open (open lists are heterogenous by definition).

1) Seems like we’re missing an optimization opportunity when we are dealing with large sparse
lists. Serialization in this case might use a bit mask to specify which items in the lists
are not null, and later encode only those items.
2) I believe if we alter IntroduceEnforcedListTypeRule to enforce list to homogeneous list
with nullable item type we might resolve issues,,
at once.


Best regards,

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message