spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Map tuple to case class in Dataset
Date Wed, 01 Jun 2016 02:37:09 GMT
Using spark-shell of 1.6.1 :

scala> case class Test(a: Int)
defined class Test

scala> Seq(1,2).toDS.map(t => Test(t)).show
+---+
|  a|
+---+
|  1|
|  2|
+---+

FYI

On Tue, May 31, 2016 at 7:35 PM, Tim Gautier <tim.gautier@gmail.com> wrote:

> 1.6.1 The exception is a null pointer exception. I'll paste the whole
> thing after I fire my cluster up again tomorrow.
>
> I take it by the responses that this is supposed to work?
>
> Anyone know when the next version is coming out? I keep running into bugs
> with 1.6.1 that are hindering my progress.
>
> On Tue, May 31, 2016, 8:21 PM Saisai Shao <sai.sai.shao@gmail.com> wrote:
>
>> It works fine in my local test, I'm using latest master, maybe this bug
>> is already fixed.
>>
>> On Wed, Jun 1, 2016 at 7:29 AM, Michael Armbrust <michael@databricks.com>
>> wrote:
>>
>>> Version of Spark? What is the exception?
>>>
>>> On Tue, May 31, 2016 at 4:17 PM, Tim Gautier <tim.gautier@gmail.com>
>>> wrote:
>>>
>>>> How should I go about mapping from say a Dataset[(Int,Int)] to a
>>>> Dataset[<case class here>]?
>>>>
>>>> I tried to use a map, but it throws exceptions:
>>>>
>>>> case class Test(a: Int)
>>>> Seq(1,2).toDS.map(t => Test(t)).show
>>>>
>>>> Thanks,
>>>> Tim
>>>>
>>>
>>>
>>

Mime
View raw message