spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeremy Freeman (JIRA)" <>
Subject [jira] [Created] (SPARK-4969) Add binaryRecords support to streaming
Date Thu, 25 Dec 2014 18:25:13 GMT
Jeremy Freeman created SPARK-4969:

             Summary: Add binaryRecords support to streaming
                 Key: SPARK-4969
             Project: Spark
          Issue Type: Improvement
          Components: PySpark, Streaming
    Affects Versions: 1.2.0
            Reporter: Jeremy Freeman
            Priority: Minor

As of Spark 1.2 there is support for loading fixed length records from flat binary files.
This is a useful way to load dense numerical array data into Spark, especially in scientific
computing applications.

We should add support for loading this same file type in Spark Streaming, both in Scala/Java
and in Python. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message