drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Hausenblas <michael.hausenb...@gmail.com>
Subject Re: First pass at a reference interpreter
Date Tue, 15 Jan 2013 01:07:37 GMT

> Given that Java6 is public updates EOL next month, I've targeted Java7.
> You probably need to update your JDK.

Good point. I suppose this should be in big red blinking letters on top of the installation
instructions.

'mvn verify' is a success now for me ... in case anyone is on MacOS 10, I've used the JDK
from [1] and works like a charm.

Cheers,
		Michael

[1] http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html

--
Michael Hausenblas
Ireland, Europe
http://mhausenblas.info/

On 14 Jan 2013, at 16:37, Jacques Nadeau <jacques.drill@gmail.com> wrote:

> Given that Java6 is public updates EOL next month, I've targeted Java7.
> You probably need to update your JDK.
> 
> Jacques
> 
> On Mon, Jan 14, 2013 at 4:31 PM, Michael Hausenblas <
> michael.hausenblas@gmail.com> wrote:
> 
>> 
>>> Be careful, there are many sharp edges.
>>> 
>>> :)
>> 
>> 
>> Alright ;)
>> 
>> So here we go: 'mvn validate' works fine for me but then ...
>> 
>> $ mvn verify
>> 
>> [INFO] prototype-parent .................................. SUCCESS [7.166s]
>> [INFO] common ............................................ FAILURE
>> [17.775s]
>> [INFO] contrib-parent .................................... SKIPPED
>> [INFO] pom ............................................... SKIPPED
>> [INFO] exec .............................................. SKIPPED
>> [INFO] java-exec ......................................... SKIPPED
>> [INFO] Logical Plan Execution Reference Implementation ... SKIPPED
>> [INFO] planner ........................................... SKIPPED
>> [INFO] sqlparser ......................................... SKIPPED
>> 
>> 
>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
>> goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile
>> (default-compile) on project common: Compilation failure
>> Failure executing javac, but could not parse the error:
>> javac: invalid target release: 1.7
>> Usage: javac <options> <source files>
>> use -help for a list of possible options
>> 
>> 
>>        at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
>>        at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>>        at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>>        at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
>>        at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
>>        at
>> org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
>>        at
>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
>>        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:319)
>>        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
>>        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
>>        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
>>        at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>        at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
>>        at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
>>        at
>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
>>        at
>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
>> Caused by: org.apache.maven.plugin.CompilationFailureException:
>> Compilation failure
>> Failure executing javac, but could not parse the error:
>> javac: invalid target release: 1.7
>> Usage: javac <options> <source files>
>> use -help for a list of possible options
>> 
>> 
>>        at
>> org.apache.maven.plugin.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:516)
>>        at
>> org.apache.maven.plugin.CompilerMojo.execute(CompilerMojo.java:114)
>>        at
>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
>>        at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
>> 
>> 
>> 
>> Cheers,
>>                Michael
>> 
>> --
>> Michael Hausenblas
>> Ireland, Europe
>> http://mhausenblas.info/
>> 
>> On 14 Jan 2013, at 16:06, Jacques Nadeau <jacques.drill@gmail.com> wrote:
>> 
>>> Be careful, there are many sharp edges.
>>> 
>>> :)
>>> 
>>> On Mon, Jan 14, 2013 at 4:02 PM, Michael Hausenblas <
>>> michael.hausenblas@gmail.com> wrote:
>>> 
>>>> 
>>>> Cool stuff, Jacques - will give it a shot ASAP!
>>>> 
>>>> Cheers,
>>>>               Michael
>>>> 
>>>> --
>>>> Michael Hausenblas
>>>> Ireland, Europe
>>>> http://mhausenblas.info/
>>>> 
>>>> On 14 Jan 2013, at 15:56, Jacques Nadeau <jacques.drill@gmail.com>
>> wrote:
>>>> 
>>>>> I've been pulling together a reference logical plan interpreter.  I'm
>>>>> working with Ted to get it inside the Drill sandbox.  For now, you can
>>>> find
>>>>> it on my repo at https://github.com/jacques-n/incubator-drill(prototype
>>>>> branch)
>>>>> 
>>>>> 
>>>>> 
>>>>> The goals of the reference interpreter are:
>>>>> 
>>>>> 
>>>>> - To provide a simple way to run a Logical Plan against some sample
>>>> data
>>>>> and get back the expected result
>>>>> - Allow work to start on the parsers while we scale up the performance
>>>>> and capabilities of the execution engine and optimizer.
>>>>> - Allow evaluation work on particular technical approaches such as
>>>>> exploring the impact of hierarchical and schema less data on query
>>>>> evaluation.
>>>>> 
>>>>> These goals do not include performance, memory handling, or
>>>>> efficiency.  Currently,
>>>>> the interpreter is a single node/thread process.  This will change
>>>> shortly
>>>>> so that it also run as a clustered process.
>>>>> 
>>>>> The entry point is inside the /sandbox/prototype/exec/ref module:
>>>>> org.apache.drill.exec.ref.ReferenceInterpreter.main();  The example
>>>> program
>>>>> utilizes two resources: simple-plan.json and donuts.json and outputs
>> data
>>>>> to /opt/data/out.json.
>>>>> 
>>>>> 
>>>>> Some of things that 'work'.
>>>>> 
>>>>> 
>>>>> - Read/write basic json.
>>>>> - ROPs (reference operators): Filter, Transform, Group, Aggregate
>>>>> (simple), Order, Union.
>>>>> - Example aggregate and basic functions including sum, count,
>> multiply,
>>>>> add, compare, equals.
>>>>> 
>>>>> Basic glossary/concepts (we'll get this on the wiki/javadocs):
>>>>> 
>>>>> 
>>>>> - LOP: Logical Operator.  An implementation agnostic data flow
>> operator
>>>>> utilized by the Logical Plan.
>>>>> - ROP: Reference Operator: A reference operator implementation that
>>>>> pairs with a LOP.
>>>>> - FunctionDefinition: A definition of a particular function.
>> Describes
>>>>> a set of aliases, an allowable set of input arguments and an interface
>>>> that
>>>>> will attempt to determine output type.
>>>>> - BasicEvaluator: An implementation of a particular non-aggregate
>>>>> expression.  Receives a record pointer at creation time. Returns a
>>>>> DataValue.
>>>>> - AggregateEvaluator: An implementation of a particular aggregating
>>>>> function.  Is provided a record pointer at creation time.  Expects
>>>> regular
>>>>> calls to addRecord() followed by a call to eval() which provides the
>>>>> aggregate value.
>>>>> - DataValue: A pointer to a particular data value.  Implementation
>>>>> classes includes things like ScalarLong, ScalarBytes, SimpleMapValue
>>>> and
>>>>> SimpleArrayValue.
>>>>> 
>>>>> The standard record iterator utilized between each ROP utilizes the
>>>>> org.apache.drill.exec.ref.RecordIterator interface.  This is somewhat
>>>>> inspired by the AttributeSource concepts from within the Lucene
>> project.
>>>>> (I'm planning to extend these concepts all the way to the individual
>>>>> DataValues.)
>>>>> 
>>>>> 
>>>>> 
>>>>> My next goals are to add tests, finish adding ROPs, add local and
>> remote
>>>>> exchange nodes (parallelization), add a bunch of documentation and
>>>> extract
>>>>> out the Execution plan as a separate intermediate representation.
>>>>> 
>>>>> 
>>>>> 
>>>>> It needs a lot more evaluators to be a true reference interpreter (as
>>>> well
>>>>> as the rest of the ROPs).  The existing ones can be utilized as
>>>> prototypes.
>>>>> Anyone interested in ripping through a bunch of additional evaluators
>> and
>>>>> associated FunctionDefinitions?
>>>> 
>>>> 
>> 
>> 


Mime
View raw message