spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Tian (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD
Date Thu, 11 Dec 2014 06:10:12 GMT

    [ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242193#comment-14242193
] 

Yi Tian commented on SPARK-4817:
--------------------------------

Hi, [~sowen]
I think the main idea of [~surq] is 
* For the people who just start to work on spark streaming (just like [~surq]), the {{print}}
api is not clear enough for them to understand that only 10 lines of data will be handled.

* In some cases, developers need a api to handle all the data and print the first few lines
of all.


> [streaming]Print the specified number of data and handle all of the elements in RDD
> -----------------------------------------------------------------------------------
>
>                 Key: SPARK-4817
>                 URL: https://issues.apache.org/jira/browse/SPARK-4817
>             Project: Spark
>          Issue Type: New Feature
>          Components: Streaming
>            Reporter: 宿荣全
>            Priority: Minor
>
> Dstream.print function:Print 10 elements and handle 11 elements.
> A new function based on Dstream.print function is presented:
> the new function:
> Print the specified number of data and handle all of the elements in RDD.
> there is a work scene:
> val dstream = stream.map->filter->mapPartitions->print
> the data after filter need update database in mapPartitions,but don't need print each
data,only need to print the top 20 for view the data processing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message