trafodion-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anoop Sharma <anoop.sha...@esgyn.com>
Subject RE: SERIALIZED option on columns
Date Tue, 27 Oct 2015 17:34:29 GMT
Additional info on this option.

Currently, this option is only used for non-aligned format tables where each
column
value is stored as one cell.

We cannot pushdown preds on aligned format rows as hbase filter since
columns within
 that row are not known to hbase.
We have plans to extend that for aligned format table either as a user
written custom
filter that can interpret aligned row or through a coprocessor.


anoop

-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, October 27, 2015 10:31 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: SERIALIZED option on columns

That is great, I was thinking that predicate pushdown would need expensive
data conversion on RS side because we could not use byte compares, thanks to
this new encoding, this is taken care off. Coolness...

-----Original Message-----
From: Anoop Sharma [mailto:anoop.sharma@esgyn.com]
Sent: Tuesday, October 27, 2015 12:26 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: SERIALIZED option on columns

it encodes columns before writing them to hbase so they could be used by
hbase to compare as bytes. Helps with pushdown of preds.
If this option is not specified, then column values are written as it.
Default is SERIALIZATION ON.

On read, we decode it back to its original value.
Same kind of encoding that we do for key columns.

Code is in exp/exp_function.cpp where it deals with encoding and decoding.

anoop


-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, October 27, 2015 10:23 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: SERIALIZED option on columns

And what is the semantic? Is it just to say that this should not be in
aligned format in a mix format mode? Or is there more to it?
Eric

-----Original Message-----
From: Anoop Sharma [mailto:anoop.sharma@esgyn.com]
Sent: Tuesday, October 27, 2015 12:17 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: SERIALIZED option on columns

Yes, it is a new option in trafodion.
It was just enabled as default.
We can update documentation about it.

anoop

-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, October 27, 2015 10:15 AM
To: dev@trafodion.incubator.apache.org
Subject: SERIALIZED option on columns

Hi all,

I am trying to understand predicate push down code. There is reference to
columns tagged “serialized”. So I went on the SQL reference manual looking
for what was the semantic of this tag. Could not find a single hit on
“Serialized” keyword. Is that a new feature? Or is the documentation needing
an update?

I can open a Jira on documentation if needed?

Eric

Mime
View raw message