spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sib <alexsib...@gmail.com>
Subject SparkSql Catalyst extending Analyzer, Error with CatalystConf
Date Fri, 13 May 2016 13:06:12 GMT
Hello,

I am trying to write a basic analyzer, by extending the catalyst analyzer
with a few extra rules.

I am getting the following error:
*""" trait CatalystConf in package catalyst cannot be accessed in package
org.apache.spark.sql.catalyst """*


In my attempt I am doing the following:

class CustomSQLContext(sc: SparkContext) extends SQLContext(sc) {
    val an = new CustomAnalyzer(Map("testRule" ->
testRule),catalog,functionRegistry,conf)
    override lazy val analyzer: Analyzer = an
  }

class CustomAnalyzer(rules: Map[String, Rule[LogicalPlan]], catalog:
Catalog, registery: FunctionRegistry, conf: CatalystConf )
    extends Analyzer( catalog, registery, conf) {
    ......
    override lazy val batches = my_batch.toSeq ++ default_batches ++ Nil
  }

Any ideas how I can pass the conf to the customAnalyzer without this error?

I tried passing it as SQLConf but get a not found error and importing
doesn't seem to work.


Thanks!



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/SparkSql-Catalyst-extending-Analyzer-Error-with-CatalystConf-tp26950.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message