spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexey Grishchenko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-9878) ReduceByKey + FullOuterJoin return 0 element if using an empty RDD
Date Tue, 01 Sep 2015 10:02:45 GMT

    [ https://issues.apache.org/jira/browse/SPARK-9878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14725117#comment-14725117
] 

Alexey Grishchenko commented on SPARK-9878:
-------------------------------------------

Not reproduced on master:
{code}
scala> println("ok :"+sc.parallelize(List((3,4),(4,5))).fullOuterJoin(sc.emptyRDD[(Int,Seq[Int])]).count)
ok :2
scala> println("ko: "+sc.parallelize(List((3,4),(4,5))).fullOuterJoin(sc.emptyRDD[(Int,Seq[Int])].reduceByKey((e1,
e2) => e1 ++ e2)).count)
ko: 2
{code}

>  ReduceByKey + FullOuterJoin return 0 element if using an empty RDD
> -------------------------------------------------------------------
>
>                 Key: SPARK-9878
>                 URL: https://issues.apache.org/jira/browse/SPARK-9878
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 1.4.0
>         Environment: linux ubuntu 64b spark-hadoop
> launched with Local[2]
>            Reporter: durand remi
>            Priority: Minor
>
> code to reproduce:
> println("ok :"+sc.parallelize(List((3,4),(4,5))).fullOuterJoin(sc.emptyRDD[(Int,Seq[Int])]).count)
> println("ko: "+sc.parallelize(List((3,4),(4,5))).fullOuterJoin(sc.emptyRDD[(Int,Seq[Int])].reduceByKey((e1,
e2) => e1 ++ e2)).count)
> what i expect: 
> ok: 2
> ko: 2
> but what i have:
> ok: 2
> ko: 0



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message