trafodion-codereview mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sureshsubbiah <...@git.apache.org>
Subject [GitHub] incubator-trafodion pull request #637: [TRAFODION-2138] Hive scan on wide ta...
Date Wed, 03 Aug 2016 20:39:53 GMT
GitHub user sureshsubbiah opened a pull request:

    https://github.com/apache/incubator-trafodion/pull/637

    [TRAFODION-2138] Hive scan on wide tables can result in lost rows or error

    The fix has three parts
    a)	RangeTailIOSize will now default to maximum length of a row, if > 16KB. 
    Each esp range will look ahead by this size. Previously this was hard coded to 16KB.
    b)	If a whole buffer does not have the start of a row, that is not an error.
    c)	We raise an error during compile if maximum row size is greater than size of IO buffer.

    We need to have the whole row (from start to finish) in a contiguous buffer to do the
rest of our logic. 
    Currently IO buffer defaults to 64 MB.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/sureshsubbiah/incubator-trafodion traf-2094

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-trafodion/pull/637.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #637
    
----
commit 6d266221b959c14319a760716939f261a2460276
Author: Suresh Subbiah <sureshs@apache.org>
Date:   2016-08-03T20:35:43Z

    [TRAFODION-2138] Hive scan on wide tables can result in lost rows or error
    
    The fix has three parts
    a)	RangeTailIOSize will now default to maximum length of a row, if > 16KB.
    Each esp range will look ahead by this size. Previously this was hard coded to 16KB.
    b)	If a whole buffer does not have the start of a row, that is not an error.
    c)	We raise an error during compile if maximum row size is greater than size of IO buffer.
    We need to have the whole row (from start to finish) in a contiguous buffer to do the
rest of our logic.
    Currently IO buffer defaults to 64 MB.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message