phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Praveen Murugesan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-990) OOM caused by order by query returning all rows
Date Thu, 19 Jun 2014 18:50:26 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037666#comment-14037666
] 

Praveen Murugesan commented on PHOENIX-990:
-------------------------------------------

[~prkommireddi] I think yes and no, it will be good to have finite control on settings, but
this can soon become over-whelming (It makes me yelp everytime i look at the list of settings
hadoop offers :p). This is something very internal to how phoenix allocates buffers, so I'm
not sure if this is a good candidate for exposure, because if exposed you expect the user
to have finite details about how phoenix does things which increases the barrier to entry
to use phoenix

That said, one thing i forgot to add to this thread is, I really am not sure of the need of
mappedbytebuffers in this case, from the code i looked at, it looked like only one thread
is operating on the file at a particular time (I might be wrong) [~giacomotaylor] Can you
elaborate the choice of using a mapped byte buffer. I'm just curious and want to learn.

> OOM caused by order by query returning all rows
> -----------------------------------------------
>
>                 Key: PHOENIX-990
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-990
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 3.0.0, 4.0.0
>            Reporter: Mujtaba Chohan
>            Assignee: Praveen Murugesan
>         Attachments: PHOENIX-990-TEST.patch, PHOENIX-990.patch
>
>
> OOM error with the following stack trace with large number of rows for query with order
by returning all rows without limit or aggregation (ex. select * from table order by col1).
Created a local perf. test to verify it when this gets fixed. Also performance.py script can
be used to generate a few mil. rows.
> Originally reported by @zenmehra.
> Stack:
> ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
> org.apache.hadoop.hbase.DoNotRetryIOException: PERFORMANCE_5000000,,1400524730456.c62cccdac8cffd098d236f5e282564bb.:
java.io.IOException: Map failed
> 	at org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:73)
> 	at org.apache.phoenix.coprocessor.ScanRegionObserver.getTopNScanner(ScanRegionObserver.java:221)
> 	at org.apache.phoenix.coprocessor.ScanRegionObserver.doPostScannerOpen(ScanRegionObserver.java:197)
> 	at org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:66)
> 	at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1316)
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2573)
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2541)
> 	at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:323)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> Caused by: java.lang.RuntimeException: java.io.IOException: Map failed
> 	at org.apache.phoenix.iterate.MappedByteBufferSortedQueue.offer(MappedByteBufferSortedQueue.java:73)
> 	at org.apache.phoenix.iterate.MappedByteBufferSortedQueue.offer(MappedByteBufferSortedQueue.java:43)
> 	at java.util.AbstractQueue.add(AbstractQueue.java:68)
> 	at org.apache.phoenix.iterate.OrderedResultIterator.getResultIterator(OrderedResultIterator.java:224)
> 	at org.apache.phoenix.iterate.OrderedResultIterator.next(OrderedResultIterator.java:169)
> 	at org.apache.phoenix.coprocessor.ScanRegionObserver.getTopNScanner(ScanRegionObserver.java:216)
> 	... 10 more
> Caused by: java.io.IOException: Map failed
> 	at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748)
> 	at org.apache.phoenix.iterate.MappedByteBufferSortedQueue$MappedByteBufferPriorityQueue.writeResult(MappedByteBufferSortedQueue.java:299)
> 	at org.apache.phoenix.iterate.MappedByteBufferSortedQueue.offer(MappedByteBufferSortedQueue.java:65)
> 	... 15 more
> Caused by: java.lang.OutOfMemoryError: Map failed
> 	at sun.nio.ch.FileChannelImpl.map0(Native Method)
> 	at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:745)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message