phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vaclav Loffelmann (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PHOENIX-965) Cannot aggregate on index with NULL values
Date Mon, 02 Jun 2014 11:43:01 GMT

     [ https://issues.apache.org/jira/browse/PHOENIX-965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vaclav Loffelmann updated PHOENIX-965:
--------------------------------------

    Attachment: nullableIndexFix.patch

> Cannot aggregate on index with NULL values
> ------------------------------------------
>
>                 Key: PHOENIX-965
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-965
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 3.0.0, 4.0.0
>            Reporter: Lukas Benes
>         Attachments: nullableIndexFix.patch
>
>
> I've found issue related to indexed NULL values (I've also applied PHOENIX-63 that writing
NULL values to index )
> CREATE TABLE tbl (id INTEGER, col INTEGER  CONSTRAINT pk PRIMARY KEY (id))
> CREATE index tbl_idx ON tbl (col, id)  /* without index works ok */
> UPSERT INTO tbl (id, col) VALUES (1, NULL) /* problematic line with NULL value */
> UPSERT INTO tbl (id, col) VALUES (2, 4)
> UPSERT INTO tbl (id, col) VALUES (3, 5)
> SELECT SUM(col) FROM tbl  
> SUM on index table with null values causes ServerAggregator exception, because empty
byte array is send to aggregate function, due to evaluate  
> expression [TO_INTEGER(PK[-1])] [\x00\x80\x00\x00\x01/0:_0/1398957414306/Put/vlen=0/ts=2]
returns true and ptr is set to EMPTY_BYTE_ARRAY, because length is 0
> https://github.com/apache/incubator-phoenix/blob/3.0/phoenix-core/src/main/java/org/apache/phoenix/expression/RowKeyColumnExpression.java#L116
> I'm not sure if it is fixable by returning false when evaluated to NULL, but here is
hotfix 
> https://github.com/tzolkincz/incubator-phoenix/commit/adf24c887d7c5248ee12c7970580c187e8f5e436#diff-6d1bc085da39a4ee2e51c36a47a33e21R117
> org.apache.phoenix.exception.PhoenixIOException: org.apache.phoenix.exception.PhoenixIOException:
org.apache.hadoop.hbase.DoNotRetryIOException: TBL_IDX,,1399026966936.08f1d45504f9f6f2c9c3f01d5af612d0.:
0
> 	at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:97)
> 	at org.apache.phoenix.iterate.ParallelIterators.getIterators(ParallelIterators.java:289)
> 	at org.apache.phoenix.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:44)
> 	at org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:66)
> 	at org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:86)
> 	at org.apache.phoenix.iterate.GroupedAggregatingResultIterator.next(GroupedAggregatingResultIterator.java:68)
> 	at org.apache.phoenix.iterate.UngroupedAggregatingResultIterator.next(UngroupedAggregatingResultIterator.java:39)
> 	at org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:732)
> 	at org.apache.phoenix.end2end.IndexSumIT.testIndexSumAggregate(IndexSumIT.java:45)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> 	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> 	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: java.util.concurrent.ExecutionException: org.apache.phoenix.exception.PhoenixIOException:
org.apache.hadoop.hbase.DoNotRetryIOException: TBL_IDX,,1399026966936.08f1d45504f9f6f2c9c3f01d5af612d0.:
0
> 	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232)
> 	at java.util.concurrent.FutureTask.get(FutureTask.java:91)
> 	at org.apache.phoenix.iterate.ParallelIterators.getIterators(ParallelIterators.java:283)
> 	... 33 more
> Caused by: org.apache.phoenix.exception.PhoenixIOException: org.apache.hadoop.hbase.DoNotRetryIOException:
TBL_IDX,,1399026966936.08f1d45504f9f6f2c9c3f01d5af612d0.: 0
> 	at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:97)
> 	at org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:57)
> 	at org.apache.phoenix.iterate.ParallelIterators$2.call(ParallelIterators.java:251)
> 	at org.apache.phoenix.iterate.ParallelIterators$2.call(ParallelIterators.java:245)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> 	at java.lang.Thread.run(Thread.java:695)
> Caused by: org.apache.hadoop.hbase.DoNotRetryIOException: org.apache.hadoop.hbase.DoNotRetryIOException:
TBL_IDX,,1399026966936.08f1d45504f9f6f2c9c3f01d5af612d0.: 0
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> 	at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
> 	at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
> 	at org.apache.hadoop.hbase.client.ServerCallable.translateException(ServerCallable.java:256)
> 	at org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:166)
> 	at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:210)
> 	at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:125)
> 	at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:120)
> 	at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:702)
> 	at org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:54)
> 	... 7 more
> Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.DoNotRetryIOException:
TBL_IDX,,1399026966936.08f1d45504f9f6f2c9c3f01d5af612d0.: 0
> 	at org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:73)
> 	at org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:68)
> 	at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1316)
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2566)
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2534)
> 	at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:323)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 0
> 	at org.apache.phoenix.schema.PDataType$IntCodec.decodeInt(PDataType.java:5709)
> 	at org.apache.phoenix.schema.PDataType$IntCodec.decodeLong(PDataType.java:5690)
> 	at org.apache.phoenix.schema.PDataType$BaseCodec.decodeLong(PDataType.java:5472)
> 	at org.apache.phoenix.expression.aggregator.NumberSumAggregator.aggregate(NumberSumAggregator.java:68)
> 	at org.apache.phoenix.expression.aggregator.ServerAggregators.aggregate(ServerAggregators.java:61)
> 	at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.doPostScannerOpen(UngroupedAggregateRegionObserver.java:277)
> 	at org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:66)
> 	... 8 more
> 	at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1012)
> 	at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:87)
> 	at com.sun.proxy.$Proxy20.openScanner(Unknown Source)
> 	at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:224)
> 	at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:126)
> 	at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:42)
> 	at org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:164)
> 	... 12 more



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message