spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam Breindel (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-17896) Dataset groupByKey + reduceGroups fails with codegen-related exception
Date Wed, 12 Oct 2016 22:25:20 GMT
Adam Breindel created SPARK-17896:
-------------------------------------

             Summary: Dataset groupByKey + reduceGroups fails with codegen-related exception
                 Key: SPARK-17896
                 URL: https://issues.apache.org/jira/browse/SPARK-17896
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.0.1
         Environment: Databricks, MacOS
            Reporter: Adam Breindel


possible regression: works on 2.0, fails on 2.0.1
following code raises exception related to wholestage codegen:

case class Zip(city:String, zip:String, state:String)

val z1 = Zip("New York", "10000", "NY")
val z2 = Zip("New York", "10001", "NY")
val z3 = Zip("Chicago", "60606", "IL")

val zips = sc.parallelize(Seq(z1, z2, z3)).toDS

zips.groupByKey(_.state).reduceGroups((z1, z2) => Zip("*", z1.zip + " " + z2.zip, z1.state)).show



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message