spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Corey Nolet <cjno...@gmail.com>
Subject Spark SQL Custom Predicate Pushdown
Date Fri, 16 Jan 2015 05:50:48 GMT
I have document storage services in Accumulo that I'd like to expose to
Spark SQL. I am able to push down predicate logic to Accumulo to have it
perform only the seeks necessary on each tablet server to grab the results
being asked for.

I'm interested in using Spark SQL to push those predicates down to the
tablet servers. Where wouldI begin my implementation? Currently I have an
input format which accepts a "query object" that gets pushed down. How
would I extract this information from the HiveContext/SQLContext to be able
to push this down?

Mime
View raw message