trafodion-codereview mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sureshsubbiah <...@git.apache.org>
Subject [GitHub] incubator-trafodion pull request #949: [TRAFODION-2475] Create table/index o...
Date Thu, 09 Feb 2017 01:05:01 GMT
Github user sureshsubbiah commented on a diff in the pull request:

    https://github.com/apache/incubator-trafodion/pull/949#discussion_r100210604
  
    --- Diff: core/sql/src/main/java/org/trafodion/sql/HBaseClient.java ---
    @@ -294,15 +296,27 @@ private ChangeFlags setDescriptors(Object[] tableOptions,
                    break ;
                case HBASE_COMPRESSION:
                    if (tableOption.equalsIgnoreCase("GZ"))
    +	       {    // throws IOException
    +		   CompressionTest.testCompression(Algorithm.GZ);
                        colDesc.setCompressionType(Algorithm.GZ);
    +	       }
                    else if (tableOption.equalsIgnoreCase("LZ4"))
    +	       {   // throws IOException
    +		   CompressionTest.testCompression(Algorithm.LZ4);
                        colDesc.setCompressionType(Algorithm.LZ4);
    +	       }
                    else if (tableOption.equalsIgnoreCase("LZO"))
    +	       {   // throws IOException
    +		   CompressionTest.testCompression(Algorithm.LZO);
                        colDesc.setCompressionType(Algorithm.LZO);
    +	       }
                    else if (tableOption.equalsIgnoreCase("NONE"))
                        colDesc.setCompressionType(Algorithm.NONE);
                    else if (tableOption.equalsIgnoreCase("SNAPPY"))
    +	       {   // throws IOException
    +		   CompressionTest.testCompression(Algorithm.SNAPPY);
                        colDesc.setCompressionType(Algorithm.SNAPPY); 
    +	       }
    --- End diff --
    
    Good catch Dave. I was confident that invalid compression options like "SNAPY" would be
caught earlier in the parse/bind stage. Now I see that validation of input arguments is not
thorough. I will add a commit to this PR by tomorrow with that change. Much validation can
be done on the C++ side before we call the create()/alter() Java function. For an allowed
compression setting usable by HBase, the only verification I know is the Java one provided
by HBase itself. Therefore this validation could be done once in Java side and results stored
in C++ side or we can do as this PR does, validate and raise error in Java side when necessary.
The method provided by HBase uses static variables to store results of previous validations,
so we will not be fully validating the same compression algorithm twice in a given session.
    
    This is how the error message looks like now.
    create table test5 (a int not null primary key, b int) hbase_options (compression = 'SNAPPY')
salt using 2 partitions ;
    
    *** ERROR[8448] Unable to access Hbase interface. Call to ExpHbaseInterface::create()
returned error HBASE_CREATE_ERROR(701). Cause: org.apache.hadoop.hbase.DoNotRetryIOException:
java.lang.RuntimeException: native snappy library not available: this version of libhadoop
was built without snappy support.
    org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:102)
    org.trafodion.sql.HBaseClient.setDescriptors(HBaseClient.java:317)
    org.trafodion.sql.HBaseClient.createk(HBaseClient.java:490) Caused by 
    java.lang.RuntimeException: native snappy library not available: this version of libhadoop
was built without snappy support.
    org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:64)
    org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:133)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message