phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prashant Kommireddi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-990) OOM caused by order by query returning all rows
Date Thu, 19 Jun 2014 18:58:25 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037676#comment-14037676
] 

Prashant Kommireddi commented on PHOENIX-990:
---------------------------------------------

It would be comfortable to not have it configurable if we can guarantee users that 128KB *always*
works. Reason hadoop has so many configs is precisely that, there is no single panacea setting
that could work for all. For example, an easy one at that, io.sort.mb can be tuned just to
improve sort performance (or sometimes just make it work if default is a lot). Users might
also want to tweak this in their tests vs production.



> OOM caused by order by query returning all rows
> -----------------------------------------------
>
>                 Key: PHOENIX-990
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-990
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 3.0.0, 4.0.0
>            Reporter: Mujtaba Chohan
>            Assignee: Praveen Murugesan
>         Attachments: PHOENIX-990-TEST.patch, PHOENIX-990.patch
>
>
> OOM error with the following stack trace with large number of rows for query with order
by returning all rows without limit or aggregation (ex. select * from table order by col1).
Created a local perf. test to verify it when this gets fixed. Also performance.py script can
be used to generate a few mil. rows.
> Originally reported by @zenmehra.
> Stack:
> ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
> org.apache.hadoop.hbase.DoNotRetryIOException: PERFORMANCE_5000000,,1400524730456.c62cccdac8cffd098d236f5e282564bb.:
java.io.IOException: Map failed
> 	at org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:73)
> 	at org.apache.phoenix.coprocessor.ScanRegionObserver.getTopNScanner(ScanRegionObserver.java:221)
> 	at org.apache.phoenix.coprocessor.ScanRegionObserver.doPostScannerOpen(ScanRegionObserver.java:197)
> 	at org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:66)
> 	at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1316)
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2573)
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2541)
> 	at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:323)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> Caused by: java.lang.RuntimeException: java.io.IOException: Map failed
> 	at org.apache.phoenix.iterate.MappedByteBufferSortedQueue.offer(MappedByteBufferSortedQueue.java:73)
> 	at org.apache.phoenix.iterate.MappedByteBufferSortedQueue.offer(MappedByteBufferSortedQueue.java:43)
> 	at java.util.AbstractQueue.add(AbstractQueue.java:68)
> 	at org.apache.phoenix.iterate.OrderedResultIterator.getResultIterator(OrderedResultIterator.java:224)
> 	at org.apache.phoenix.iterate.OrderedResultIterator.next(OrderedResultIterator.java:169)
> 	at org.apache.phoenix.coprocessor.ScanRegionObserver.getTopNScanner(ScanRegionObserver.java:216)
> 	... 10 more
> Caused by: java.io.IOException: Map failed
> 	at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748)
> 	at org.apache.phoenix.iterate.MappedByteBufferSortedQueue$MappedByteBufferPriorityQueue.writeResult(MappedByteBufferSortedQueue.java:299)
> 	at org.apache.phoenix.iterate.MappedByteBufferSortedQueue.offer(MappedByteBufferSortedQueue.java:65)
> 	... 15 more
> Caused by: java.lang.OutOfMemoryError: Map failed
> 	at sun.nio.ch.FileChannelImpl.map0(Native Method)
> 	at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:745)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message