spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joseph K. Bradley (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-15255) RDD name from DataFrame op should not include full local relation data
Date Tue, 10 May 2016 19:12:13 GMT
Joseph K. Bradley created SPARK-15255:
-----------------------------------------

             Summary: RDD name from DataFrame op should not include full local relation data
                 Key: SPARK-15255
                 URL: https://issues.apache.org/jira/browse/SPARK-15255
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 2.0.0
            Reporter: Joseph K. Bradley
            Priority: Minor


Currently, if you create a DataFrame from local data, do some operations with it, and cache
it, then the name of the RDD in the "Storage" tab in the Spark UI will contain the entire
local relation's data.  This is not scalable and can cause the browser to become unresponsive.

I'd propose there be a limit on the size of the data to display.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message