spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Herman van Hövell tot Westerflier <hvanhov...@databricks.com>
Subject Re: encoders for more complex types
Date Thu, 27 Oct 2016 20:02:18 GMT
What kind of difficulties are you experiencing?

On Thu, Oct 27, 2016 at 9:57 PM, Koert Kuipers <koert@tresata.com> wrote:

> i have been pushing my luck a bit and started using ExpressionEncoder for
> more complex types like sequences of case classes etc. (where the case
> classes only had primitives and Strings).
>
> it all seems to work but i think the wheels come off in certain cases in
> the code generation. i guess this is not unexpected, after all what i am
> doing is not yet supported.
>
> is there a planned path forward to support more complex types with
> encoders? it would be nice if we can at least support all types that
> spark-sql supports in general for DataFrame.
>
> best, koert
>

Mime
View raw message