spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <>
Subject [jira] [Assigned] (SPARK-17350) Disable default use of KryoSerializer in Thrift Server
Date Thu, 01 Sep 2016 00:12:20 GMT


Apache Spark reassigned SPARK-17350:

    Assignee: Josh Rosen  (was: Apache Spark)

> Disable default use of KryoSerializer in Thrift Server
> ------------------------------------------------------
>                 Key: SPARK-17350
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>            Reporter: Josh Rosen
>            Assignee: Josh Rosen
> In SPARK-4761 (December 2014) we enabled Kryo serialization by default in the Spark Thrift
Server. However, I don't think that the original rationale for doing this still holds as all
Spark SQL serialization should now be performed via efficient encoders and our UnsafeRow format.
In addition, the use of Kryo as the default serializer can introduce performance problems
because the creation of new KryoSerializer instances is expensive and we haven't performed
instance-reuse optimizations in several code paths (including DirectTaskResult deserialization).
Given all of this, I propose to revert back to using JavaSerializer as the default serializer
in the Thrift Server.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message