flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Flink Jira Bot (Jira)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-19363) Code of split method grows beyond 64 KB
Date Sun, 30 May 2021 23:14:01 GMT

     [ https://issues.apache.org/jira/browse/FLINK-19363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Flink Jira Bot updated FLINK-19363:
-----------------------------------
      Labels: auto-deprioritized-major  (was: stale-major)
    Priority: Minor  (was: Major)

This issue was labeled "stale-major" 7 ago and has not received any updates so it is being
deprioritized. If this ticket is actually Major, please raise the priority and ask a committer
to assign you the issue or revive the public discussion.


> Code of split method grows beyond 64 KB
> ---------------------------------------
>
>                 Key: FLINK-19363
>                 URL: https://issues.apache.org/jira/browse/FLINK-19363
>             Project: Flink
>          Issue Type: Improvement
>          Components: Table SQL / Runtime
>    Affects Versions: 1.11.0
>            Reporter: hailong wang
>            Priority: Minor
>              Labels: auto-deprioritized-major
>             Fix For: 1.14.0
>
>
> For now, If the total size beyond 64kb, we will split one field into one field. But when
sql is complicated, one field size will grow 64kb.
> {code:java}
> Caused by: org.codehaus.janino.InternalCompilerException: Compiling "StreamExecCalc$4436":
Code of method "split$4435$(LStreamExecCalc$4436;)V" of class "StreamExecCalc$4436" grows
beyond 64 KBCaused by: org.codehaus.janino.InternalCompilerException: Compiling "StreamExecCalc$4436":
Code of method "split$4435$(LStreamExecCalc$4436;)V" of class "StreamExecCalc$4436" grows
beyond 64 KB at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:382) at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:237)
at org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:465) at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:216)
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:207) at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:80)
at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:75) at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:78)
... 24 moreCaused by: org.codehaus.janino.InternalCompilerException: Code of method "split$4435$(LStreamExecCalc$4436;)V"
of class "StreamExecCalc$4436" grows beyond 64 KB at org.codehaus.janino.CodeContext.makeSpace(CodeContext.java:1009)
at org.codehaus.janino.CodeContext.write(CodeContext.java:918) at org.codehaus.janino.CodeContext.writeBranch(CodeContext.java:1038)
> {code}
> By this case, we should split one field to multi functions.
> My solution is as follow,
> {code:java}
> override def visitCall(call: RexCall): GeneratedExpression = {
>   val resultType = FlinkTypeFactory.toLogicalType(call.getType)
>   // convert operands and help giving untyped NULL literals a type
>   val operands = call.getOperands.zipWithIndex.map {
>     // this helps e.g. for AS(null)
>     // we might need to extend this logic in case some rules do not create typed NULLs
>     case (operandLiteral: RexLiteral, 0) if
>     operandLiteral.getType.getSqlTypeName == SqlTypeName.NULL &&
>       call.getOperator.getReturnTypeInference == ReturnTypes.ARG0 =>
>       generateNullLiteral(resultType, ctx.nullCheck)
>     case (o@_, _) => o.accept(this)
>   }
>   // when function is too large, we split it.
>   val exp = generateCallExpression(ctx, call.getOperator, operands, resultType)
>   if (exp.code.length > maxGeneratedCodeLength) {
>     ctx.setCodeSplit()
>     val methodName = newName("callSplit")
>     val resultTypeTerm = boxedTypeTermForType(exp.resultType)
>     val callResultTerm = ctx.addReusableLocalVariable(resultTypeTerm, "callSplitResultTerm")
>     val callNullResultTerm = ctx.addReusableLocalVariable("boolean", "callSplitNullResultTerm")
>     val method =
>       s"""
>          |private void $methodName() throws Exception {
>          |  ${exp.code}
>          |  ${callResultTerm}= ${exp.resultTerm};
>          |  ${callNullResultTerm}= ${exp.nullTerm};
>          |}
>          |""".stripMargin
>     val methodCode =
>       s"""
>          |$methodName();
>          |""".stripMargin
>     ctx.addReusableMember(method)
>     return new GeneratedExpression(callResultTerm, callNullResultTerm, methodCode, exp.resultType)
>   }
>   exp
> }
> {code}
> When function beyong maxGeneratedCodeLength, we split it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message