spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacob Eisinger <eisinger.ja...@gmail.com>
Subject Re: RDD operation examples with data?
Date Thu, 31 Jul 2014 20:30:04 GMT
I would check out the source examples on Spark's Github:
https://github.com/apache/spark/tree/master/examples/src/main/scala/org/apache/spark/examples

And, Zhen He put together a great web page with summaries and examples of
each function:
http://apache-spark-user-list.1001560.n3.nabble.com/A-new-resource-for-getting-examples-of-Spark-RDD-API-calls-td5529.html

Hope this helps!

Jacob


On Thu, Jul 31, 2014 at 3:00 PM, Chris Curtin <curtin.chris@gmail.com>
wrote:

> Hi,
>
> I'm learning Spark and I am confused about when to use the many different
> operations on RDDs. Does anyone have any examples which show example inputs
> and resulting outputs for the various RDD operations and if the operation
> takes an Function a simple example of the code?
>
> For example, something like this for flatMap
>
> One row -> "the quick brown fox"
>
> Passed to:
>
> JavaRDD<String> words = lines.flatMap(new FlatMapFunction<String, String>()
{
>       @Override
>       public Iterable<String> call(String s) {
>         return Arrays.asList(SPACE.split(s));
>       }
>     });
>
> When completed: words would contain
> "the"
> "quick"
> "brown"
> "fox"
>
> (Yes this one is pretty obvious but some of the others aren't).
>
> If such examples don't exist, is there a shared wiki or someplace we could start building
one?
>
> Thanks,
>
> Chris
>
>

Mime
View raw message