Yeah, my test was entirely focused on compile time. In the case of
simple queries, the compile time differences would be important. In
larger queries, this would be less of an issue.
j
On Mon, May 20, 2013 at 8:56 AM, Lisen Mu <immars@gmail.com> wrote:
> Jacques,
>
> I've tried Janino now, with your
> org.apache.drill.exec.compile.TestClassCompilationTypes,
>
> One thing is that the compilation time dominates the benchmark. If we
> compile once and benchmark evaluate() only, the performance of class
> compiled by JDK compiler is comparable with the class compiled by Janino's
> embed compiler. JDK compiler seems to invoke javac which is more
> heavyweight than Janino.
>
> Another thing is that evalute() of ExpressionEvaluator by both Janino and
> JDK implementation need to do reflect invocation once. I assume that in the
> future utilization in drill, we would have several limited type of
> evaluate() method signature; so by using SimpleCompiler to compile the
> expression directly into a subclass of our Evaluator interface, we can
> fully avoid reflection.
>
> Here is my benchmark result:
>
> Janino: 882 micros. JDK: 1641 micros. Direct: 41 micros.
>
> Please see the
> https://github.com/immars/incubator-drill/commit/34c0c3db7287904dbfdef9751e41e252b264dc88
>
>
>
>
> On Sun, Apr 28, 2013 at 10:25 AM, Ted Dunning <ted.dunning@gmail.com> wrote:
>
>> Is the actual execution time after warming up the JIT the same?
>>
>>
>> On Fri, Apr 26, 2013 at 8:16 PM, Jacques Nadeau <jacques@apache.org>
>> wrote:
>>
>> > I'm working on some of the bytecode stuff for Drill and found some
>> > interesting random testing data:
>> >
>> > Java6 adds a new JavaCompiler API. Janino supports utilizing that
>> > instead of its internal parsing. The benefit of this is that all the
>> > JDK 5 stuff like generics are fully supported. Since that seemed
>> > nice, i figured we could use that.
>> >
>> > In initial testing, I've actually found it substantially slower than
>> > the Janino compiler. I ran some crappy microbenchmarks just to see
>> > why my stuff was going so slow. Running a basic comparison test of
>> > the simple ExpressionEvaluator example utilizing Janino versus JDK
>> > showed an average compile and single invocation time of ~18000 micros
>> > on JDK versus ~950 micros on Janino.
>> >
>> > Has anyone else experimented with this/experienced this?
>> >
>> > Example block below (straight from Janino docs).
>> >
>> > ExpressionEvaluator ee = new ExpressionEvaluator("c > d ? c : d",
>> > // expression
>> > int.class, // expressionType
>> > new String[] { "c", "d" }, // parameterNames
>> > new Class[] { int.class, int.class } // parameterTypes
>> > );
>> >
>> > // Evaluate it with varying parameter values; very fast.
>> > return (Integer) ee.evaluate(new Object[] { // parameterValues
>> > new Integer(10), new Integer(11), });
>> >
>>
|