spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dongjoon Hyun (JIRA)" <>
Subject [jira] [Closed] (SPARK-16713) Limit codegen method size to 8KB
Date Thu, 09 Feb 2017 21:53:41 GMT


Dongjoon Hyun closed SPARK-16713.
    Resolution: Won't Fix

According to the discussion on PR, I think we can close this as WON'T FIX for now.

> Limit codegen method size to 8KB
> --------------------------------
>                 Key: SPARK-16713
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core, SQL
>    Affects Versions: 2.0.0
>            Reporter: Qifan Pu
> Ideally, we would wish codegen methods to be less than 8KB for bytecode size. Beyond
8K JIT won't compile and can cause performance degradation. We have seen this for queries
with wide schema (30+ fields), where agg_doAggregateWithKeys() can be more than 8K. This is
also a major reason for performance regression when we enable fash aggregate hashmap (such
as using VectorizedHashMapGenerator.scala).

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message