trafodion-codereview mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From DaveBirdsall <...@git.apache.org>
Subject [GitHub] incubator-trafodion pull request #929: [TRAFODION-2455] Add retry to row cou...
Date Wed, 25 Jan 2017 22:47:49 GMT
GitHub user DaveBirdsall opened a pull request:

    https://github.com/apache/incubator-trafodion/pull/929

    [TRAFODION-2455] Add retry to row count estimation logic

    This set of changes does the following:
    
    1. Changes the stack from NATable::estimateHBaseRowCount on down to return detailed error
information about any failure.
    2. Changes NATable::estimateHBaseRowCount to return a rowcount of 100 million instead
of zero when an error occurs. It is safer to overestimate the size of an object than to underestimate
it.
    3. Changes UPDATE STATISTICS to give an error 9252 "Unable to get row count estimate:
Error code $0int0, detail $1int1. Exception info (if any): $0~string0" when an error occurs
in or underneath NATable::estimateHBaseRowCount instead of using a rowcount estimate of zero.
The information in this message gives details such as what error path was taken, and any Java
exception information that may be pertinent.
    4. Adds a retry loop to HBaseClient.java method estimateRowCount so that we retry if we
encounter a FileNotFoundException. UPDATE STATISTICS will do up to 4 minutes worth of accumulated
retries; normal compilation will do up to 5 seconds worth of accumulated retries. Wait times
for retries start out at 2 seconds, doubling until topping out at 30 seconds.
    5. Adds timestamps to messages in update statistics logging in local time. To get local
time, I needed to fix a bug in the monitor (process.cxx) that was incorrectly setting an environment
variable TZ to the empty string when TZ was not defined in the monitor itself.
    
    Note: The fix in process.cxx might (or might not!) fix a similar bug in DTM logging where
local timestamps are desired but UTC timestamps are produced.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/DaveBirdsall/incubator-trafodion Trafodion2455x

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-trafodion/pull/929.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #929
    
----
commit 932c219f0fd2a49e1c53a666a1dcbcb3578c4f91
Author: Dave Birdsall <dave.birdsall@esgyn.com>
Date:   2017-01-19T20:10:33Z

    [TRAFODION-2440] Add retry to row count estimation logic

commit f1653636dd6d411fafe79697ee75c8763ff8edf2
Author: Dave Birdsall <dbirdsall@apache.org>
Date:   2017-01-24T00:09:08Z

    Comment change

commit dbc6c0876ca5af07b63d68f388bad07195d106a5
Author: Dave Birdsall <dbirdsall@apache.org>
Date:   2017-01-25T22:40:56Z

    [TRAFODION-2455] More refinements to row count estimation retry logic

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message