spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: rdd conversion
Date Mon, 26 Oct 2015 17:09:58 GMT
bq.  t = new Tuple2 <Integer, ArrayList<String>> (entry.getKey(),
entry.getValue());

The return statement is outside the loop.
That was why you got one RDD.

On Mon, Oct 26, 2015 at 9:40 AM, Yasemin Kaya <godot85@gmail.com> wrote:

> Hi,
>
> I have *JavaRDD<List<Map<Integer, ArrayList<String>>>>* and I
want to
> convert every map to pairrdd, i mean
> * JavaPairRDD<Integer,ArrayList<String>>. *
>
> There is a loop in list to get the indexed map, when I write code below,
> it returns me only one rdd.
>
> JavaPairRDD<Integer,ArrayList<String>> mapToRDD =
>  IdMapValues.mapToPair(new
> PairFunction<List<Map<Integer,ArrayList<String>>>, Integer,
> ArrayList<String>>() {
>
> @Override
> public Tuple2<Integer, ArrayList<String>> call(
> List<Map<Integer, ArrayList<String>>> arg0)
> throws Exception {
> Tuple2<Integer, ArrayList<String>> t = null;
> for(int i=0; i<arg0.size(); ++i){
> for (Map.Entry<Integer, ArrayList<String>> entry :arg0.get(i).entrySet())
{
> t = new Tuple2 <Integer, ArrayList<String>> (entry.getKey(),
> entry.getValue());
> }
> }
>
> return t;
> }
> });
>
> As you can see i am using java. Give me some clue .. Thanks.
>
> Best,
> yasemin
>
> --
> hiç ender hiç
>

Mime
View raw message