spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Holden Karau <hol...@pigscanfly.ca>
Subject Re: scala.MatchError: scala.collection.immutable.Range.Inclusive from catalyst.ScalaReflection.serializerFor?
Date Mon, 09 Jan 2017 09:12:06 GMT
If you want to check if it's your modifications or just in mainline, you
can always just checkout mainline or stash your current changes to rebuild
(this is something I do pretty often when I run into bugs I don't think I
would have introduced).

On Mon, Jan 9, 2017 at 1:01 AM Liang-Chi Hsieh <viirya@gmail.com> wrote:

>
>
> Hi,
>
>
>
> As Seq(0 to 8) is:
>
>
>
> scala> Seq(0 to 8)
>
> res1: Seq[scala.collection.immutable.Range.Inclusive] = List(Range(0, 1, 2,
>
> 3, 4, 5, 6, 7, 8))
>
>
>
> Do you actually want to create a Dataset of Range? If so, I think currently
>
> ScalaReflection which the encoder relies doesn't support Range.
>
>
>
>
>
>
>
> Jacek Laskowski wrote
>
> > Hi,
>
> >
>
> > Just got this this morning using the fresh build of Spark
>
> > 2.2.0-SNAPSHOT (with a few local modifications):
>
> >
>
> > scala> Seq(0 to 8).toDF
>
> > scala.MatchError: scala.collection.immutable.Range.Inclusive (of class
>
> > scala.reflect.internal.Types$ClassNoArgsTypeRef)
>
> >   at
>
> >
> org.apache.spark.sql.catalyst.ScalaReflection$.org$apache$spark$sql$catalyst$ScalaReflection$$serializerFor(ScalaReflection.scala:520)
>
> >   at
>
> >
> org.apache.spark.sql.catalyst.ScalaReflection$.serializerFor(ScalaReflection.scala:463)
>
> >   at
>
> >
> org.apache.spark.sql.catalyst.encoders.ExpressionEncoder$.apply(ExpressionEncoder.scala:71)
>
> >   at
>
> >
> org.apache.spark.sql.SQLImplicits.newIntSequenceEncoder(SQLImplicits.scala:168)
>
> >   ... 48 elided
>
> >
>
> > Is this something I've introduced, a known issue or a bug?
>
> >
>
> > ./bin/spark-shell --version
>
> > Welcome to
>
> >       ____              __
>
> >      / __/__  ___ _____/ /__
>
> >     _\ \/ _ \/ _ `/ __/  '_/
>
> >    /___/ .__/\_,_/_/ /_/\_\   version 2.2.0-SNAPSHOT
>
> >       /_/
>
> >
>
> > Using Scala version 2.11.8, Java HotSpot(TM) 64-Bit Server VM, 1.8.0_112
>
> > Branch master
>
> > Compiled by user jacek on 2017-01-09T05:01:47Z
>
> > Revision 19d9d4c855eab8f647a5ec66b079172de81221d0
>
> > Url https://github.com/apache/spark.git
>
> > Type --help for more information.
>
> >
>
> > Pozdrawiam,
>
> > Jacek Laskowski
>
> > ----
>
> > https://medium.com/@jaceklaskowski/
>
> > Mastering Apache Spark 2.0 https://bit.ly/mastering-apache-spark
>
> > Follow me at https://twitter.com/jaceklaskowski
>
> >
>
> > ---------------------------------------------------------------------
>
> > To unsubscribe e-mail:
>
>
>
> > dev-unsubscribe@.apache
>
>
>
>
>
>
>
>
>
>
>
> -----
>
> Liang-Chi Hsieh | @viirya
>
> Spark Technology Center
>
> http://www.spark.tc/
>
> --
>
> View this message in context:
> http://apache-spark-developers-list.1001551.n3.nabble.com/scala-MatchError-scala-collection-immutable-Range-Inclusive-from-catalyst-ScalaReflection-serializer-tp20520p20522.html
>
> Sent from the Apache Spark Developers List mailing list archive at
> Nabble.com.
>
>
>
> ---------------------------------------------------------------------
>
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
>
>

Mime
View raw message