drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: Benchmarks for Apache Drill
Date Tue, 18 Aug 2015 04:52:07 GMT
On Mon, Aug 17, 2015 at 8:30 PM, Andrew Brust <
andrew.brust@bluebadgeinsights.com> wrote:

> Thanks!  Amazing how much that reminds me of writing .NET CLR functions
> and aggregates for SQL Server, something I've covered in our SQL Server
> book for the last 10 years.

It is similar, I think, particularly with regard to the annotation style. I
imagine that there may be some changes here and there to the style,
particularly in the semantic and lexical constraints on UDF functions in
Drill (which as you point out is considerably less mature than SQL Server).

> Meanwhile, and forgive me if I'm being thick, but how does that
> architecture lend itself to vectorization of the code?

Well, the code that you see isn't the code that runs.

What Drill does is use your compiled code to find the annotations to
understand the code. Then it uses the source to generate the actual code
that is run. But then because your code is inserted in-line with all the
other code in the query (with appropriate lexical constraints), the JIT
optimizer can see everything it needs in order to heavily rewrite your
code.  For simple operations like additions and such, the optimizer can
even insert vectorized code. Drill can also have special purpose operators
that recognize the potential for vectorization and insert those vectorized
operators as practical.

> -----Original Message-----
> From: Ted Dunning [mailto:ted.dunning@gmail.com]
> Sent: Monday, August 17, 2015 3:37 AM
> To: user <user@drill.apache.org>
> Subject: Re: Benchmarks for Apache Drill
> Docs:
> https://drill.apache.org/docs/develop-custom-functions-introduction/
> Some usable examples:
> https://github.com/mapr-demos/simple-drill-functions

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message