spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Youssef Hatem (JIRA)" <>
Subject [jira] [Commented] (SPARK-6320) Adding new query plan strategy to SQLContext
Date Tue, 17 Mar 2015 15:10:38 GMT


Youssef Hatem commented on SPARK-6320:

Thank you Michael for your response,

Actually I am trying to extend SQL syntax with custom constructs which would require altering
the workflow of {{SparkSQL}} all the way starting from the lexer and ending with the optimization
and physical planning.

I thought this should be possible without having to extend many classes. However not being
able to use {{planLater}} forces me to do seemingly unnecessary extension of {{SparkPlanner}}
specially that there seem to be some logic to handle these scenarios (i.e. the {{extraStrategies}}

> Adding new query plan strategy to SQLContext
> --------------------------------------------
>                 Key: SPARK-6320
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.3.0
>            Reporter: Youssef Hatem
>            Priority: Minor
> Hi,
> I would like to add a new strategy to {{SQLContext}}. To do this I created a new class
which extends {{Strategy}}. In my new class I need to call {{planLater}} function. However
this method is defined in {{SparkPlanner}} (which itself inherits the method from {{QueryPlanner}}).
> To my knowledge the only way to make {{planLater}} function visible to my new strategy
is to define my strategy inside another class that extends {{SparkPlanner}} and inherits {{planLater}}
as a result, by doing so I will have to extend the {{SQLContext}} such that I can override
the {{planner}} field with the new {{Planner}} class I created.
> It seems that this is a design problem because adding a new strategy seems to require
extending {{SQLContext}} (unless I am doing it wrong and there is a better way to do it).
> Thanks a lot,
> Youssef

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message