drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Altekruse (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-315) Reading only select columns from a parquet file
Date Mon, 02 Dec 2013 04:58:36 GMT

    [ https://issues.apache.org/jira/browse/DRILL-315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13836242#comment-13836242

Jason Altekruse commented on DRILL-315:

I actually just realized what that buffer was being used for. I was using it for debugging,
there was no permanent use for it. I was getting a thrift error when I had made the changes,
so what I did was made that temporary buffer to see what was stored in the contents of the
buffer and compared it against what I was supposed to be seeing by reverting back to the old
code and inserting the same debugging hack. It was just a stupid way to get around not being
able to see the bytes in the buffer within the debugging environment, before thift tried reading
them and threw an exception. That also explains why I re-created the buffer at the beginning,
because once the bytes were read into my debugging buffer they were not able to be re-read
out of the buffer by thrift, as there is no seek-backwards method for that type of stream.

This is more than needed to be said, because jacques fixed it, but we were looking at the
code together and I couldn't remember for the life of me why it was there.

I will create a modified patch with only the related changes.

> Reading only select columns from a parquet file
> -----------------------------------------------
>                 Key: DRILL-315
>                 URL: https://issues.apache.org/jira/browse/DRILL-315
>             Project: Apache Drill
>          Issue Type: New Feature
>            Reporter: Jason Altekruse
>            Assignee: Jason Altekruse
>         Attachments: 0001-Parquet-files-now-read-by-column-rather-than-buffer-.patch

This message was sent by Atlassian JIRA

View raw message