spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sib <>
Subject SparkSql Catalyst extending Analyzer, Error with CatalystConf
Date Fri, 13 May 2016 13:06:12 GMT

I am trying to write a basic analyzer, by extending the catalyst analyzer
with a few extra rules.

I am getting the following error:
*""" trait CatalystConf in package catalyst cannot be accessed in package
org.apache.spark.sql.catalyst """*

In my attempt I am doing the following:

class CustomSQLContext(sc: SparkContext) extends SQLContext(sc) {
    val an = new CustomAnalyzer(Map("testRule" ->
    override lazy val analyzer: Analyzer = an

class CustomAnalyzer(rules: Map[String, Rule[LogicalPlan]], catalog:
Catalog, registery: FunctionRegistry, conf: CatalystConf )
    extends Analyzer( catalog, registery, conf) {
    override lazy val batches = my_batch.toSeq ++ default_batches ++ Nil

Any ideas how I can pass the conf to the customAnalyzer without this error?

I tried passing it as SQLConf but get a not found error and importing
doesn't seem to work.


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message