hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <>
Subject [jira] [Commented] (HIVE-11595) refactor ORC footer reading to make it usable from outside
Date Thu, 27 Aug 2015 20:23:47 GMT


Prasanth Jayachandran commented on HIVE-11595:

Mostly looks good. I am concerned about having references to two copies of footer (footerBuffer
and fullFooterBuffer). I am guessing footerBuffer is subset of fullFooterBuffer (includes
metadata + ps). Can we store the postscript length and footer length in the FileMetaInfo?
So that, we can seek to postscript length - footer length and read footer length bytes to
extract the footer alone. 

> refactor ORC footer reading to make it usable from outside
> ----------------------------------------------------------
>                 Key: HIVE-11595
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-10595.patch, HIVE-11595.01.patch, HIVE-11595.02.patch, HIVE-11595.03.patch
> If ORC footer is read from cache, we want to parse it without having the reader, opening
a file, etc. I thought it would be as simple as protobuf parseFrom bytes, but apparently there's
bunch of stuff going on there. It needs to be accessible via something like parseFrom(ByteBuffer),
or similar.

This message was sent by Atlassian JIRA

View raw message