drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacq...@apache.org>
Subject Re: Janino performance versus javax.tools.JavaCompiler
Date Mon, 20 May 2013 18:23:21 GMT
Yeah, my test was entirely focused on compile time.  In the case of
simple queries, the compile time differences would be important.  In
larger queries, this would be less of an issue.

j


On Mon, May 20, 2013 at 8:56 AM, Lisen Mu <immars@gmail.com> wrote:
> Jacques,
>
> I've tried Janino now, with your
> org.apache.drill.exec.compile.TestClassCompilationTypes,
>
> One thing is that the compilation time dominates the benchmark. If we
> compile once and benchmark evaluate() only, the performance of class
> compiled by JDK compiler is comparable with the class compiled by Janino's
> embed compiler. JDK compiler seems to invoke javac which is more
> heavyweight than Janino.
>
> Another thing is that evalute() of ExpressionEvaluator by both Janino and
> JDK implementation need to do reflect invocation once. I assume that in the
> future utilization in drill, we would have several limited type of
> evaluate() method signature; so by using SimpleCompiler to compile the
> expression directly into a subclass of our Evaluator interface, we can
> fully avoid reflection.
>
> Here is my benchmark result:
>
> Janino: 882 micros.  JDK: 1641 micros. Direct: 41 micros.
>
> Please see the
> https://github.com/immars/incubator-drill/commit/34c0c3db7287904dbfdef9751e41e252b264dc88
>
>
>
>
> On Sun, Apr 28, 2013 at 10:25 AM, Ted Dunning <ted.dunning@gmail.com> wrote:
>
>> Is the actual execution time after warming up the JIT the same?
>>
>>
>> On Fri, Apr 26, 2013 at 8:16 PM, Jacques Nadeau <jacques@apache.org>
>> wrote:
>>
>> > I'm working on some of the bytecode stuff for Drill and found some
>> > interesting random testing data:
>> >
>> > Java6 adds a new JavaCompiler API.  Janino supports utilizing that
>> > instead of its internal parsing.  The benefit of this is that all the
>> > JDK 5 stuff like generics are fully supported.  Since that seemed
>> > nice, i figured we could use that.
>> >
>> > In initial testing, I've actually found it substantially slower than
>> > the Janino compiler.  I ran some crappy microbenchmarks just to see
>> > why my stuff was going so slow.  Running a basic comparison test of
>> > the simple ExpressionEvaluator example utilizing Janino versus JDK
>> > showed an average compile and single invocation time of ~18000 micros
>> > on JDK versus ~950 micros on Janino.
>> >
>> > Has anyone else experimented with this/experienced this?
>> >
>> > Example block below (straight from Janino docs).
>> >
>> >      ExpressionEvaluator ee = new ExpressionEvaluator("c > d ? c : d",
>> > // expression
>> >         int.class, // expressionType
>> >         new String[] { "c", "d" }, // parameterNames
>> >         new Class[] { int.class, int.class } // parameterTypes
>> >     );
>> >
>> >     // Evaluate it with varying parameter values; very fast.
>> >     return  (Integer) ee.evaluate(new Object[] { // parameterValues
>> >         new Integer(10), new Integer(11), });
>> >
>>

Mime
View raw message