spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Imran Rashid (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-6190) create LargeByteBuffer abstraction for eliminating 2GB limit on blocks
Date Thu, 05 Mar 2015 18:09:38 GMT
Imran Rashid created SPARK-6190:
-----------------------------------

             Summary: create LargeByteBuffer abstraction for eliminating 2GB limit on blocks
                 Key: SPARK-6190
                 URL: https://issues.apache.org/jira/browse/SPARK-6190
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
            Reporter: Imran Rashid
            Assignee: Imran Rashid


A key component in eliminating the 2GB limit on blocks is creating a proper abstraction for
storing more than 2GB.  Currently spark is limited by a reliance on nio ByteBuffer and netty
ByteBuf, both of which are limited at 2GB.  This task will introduce the new abstraction and
the relevant implementation and utilities, without effecting the existing implementation at
all.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message