spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <>
Subject [jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB
Date Sat, 21 May 2016 23:09:12 GMT


Apache Spark commented on SPARK-15285:

User 'kiszk' has created a pull request for this issue:

> Generated SpecificSafeProjection.apply method grows beyond 64 KB
> ----------------------------------------------------------------
>                 Key: SPARK-15285
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.6.1, 2.0.0
>            Reporter: Konstantin Shaposhnikov
> The following code snippet results in 
> {noformat}
>  org.codehaus.janino.JaninoRuntimeException: Code of method "(Ljava/lang/Object;)Ljava/lang/Object;"
of class "org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificSafeProjection"
grows beyond 64 KB
>   at org.codehaus.janino.CodeContext.makeSpace(
> {noformat}
> {code}
> case class S100(s1:String="1", s2:String="2", s3:String="3", s4:String="4", s5:String="5",
s6:String="6", s7:String="7", s8:String="8", s9:String="9", s10:String="10", s11:String="11",
s12:String="12", s13:String="13", s14:String="14", s15:String="15", s16:String="16", s17:String="17",
s18:String="18", s19:String="19", s20:String="20", s21:String="21", s22:String="22", s23:String="23",
s24:String="24", s25:String="25", s26:String="26", s27:String="27", s28:String="28", s29:String="29",
s30:String="30", s31:String="31", s32:String="32", s33:String="33", s34:String="34", s35:String="35",
s36:String="36", s37:String="37", s38:String="38", s39:String="39", s40:String="40", s41:String="41",
s42:String="42", s43:String="43", s44:String="44", s45:String="45", s46:String="46", s47:String="47",
s48:String="48", s49:String="49", s50:String="50", s51:String="51", s52:String="52", s53:String="53",
s54:String="54", s55:String="55", s56:String="56", s57:String="57", s58:String="58", s59:String="59",
s60:String="60", s61:String="61", s62:String="62", s63:String="63", s64:String="64", s65:String="65",
s66:String="66", s67:String="67", s68:String="68", s69:String="69", s70:String="70", s71:String="71",
s72:String="72", s73:String="73", s74:String="74", s75:String="75", s76:String="76", s77:String="77",
s78:String="78", s79:String="79", s80:String="80", s81:String="81", s82:String="82", s83:String="83",
s84:String="84", s85:String="85", s86:String="86", s87:String="87", s88:String="88", s89:String="89",
s90:String="90", s91:String="91", s92:String="92", s93:String="93", s94:String="94", s95:String="95",
s96:String="96", s97:String="97", s98:String="98", s99:String="99", s100:String="100")
> case class S(s1: S100=S100(), s2: S100=S100(), s3: S100=S100(), s4: S100=S100(), s5:
S100=S100(), s6: S100=S100(), s7: S100=S100(), s8: S100=S100(), s9: S100=S100(), s10: S100=S100())
> val ds = Seq(S(),S(),S()).toDS
> {code}
> I could reproduce this with Spark built from 1.6 branch and with

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message