spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roberto Franchini <ro.franch...@gmail.com>
Subject OrientDB through JDBC: query field names wrapped by double quote
Date Tue, 09 Aug 2016 12:36:07 GMT
Hi to all,
I'm the maintainer of the JDBC driver OrientDB.

We are trying to fetch data to Spark from an orientDB using theJDBC driver.

I'm facing some issues:


To gather metadata spark performs a "test query" of this form: select
* from TABLE_NAME whre 1=0
For this case, I write a workaround inside the driver, getting rid of
where 1=0 and replaging it with LIMIT 1.

After that query, it then performs a query with each field wrapped by
double quote:

SELECT "stringKey", "intKey" FROM Item

In orientDB's SQL dialect a double quote means a string value, so for
each record of the result set it will return stringkey and intKey as
vaules.

row 1: stringKey:strinKey, intKey:intKey
row 2: stringKey:strinKey, intKey:intKey
row 3: stringKey:strinKey, intKey:intKey
....


Is there a  way to configure SqlContext to avoid the double quoting of
fields names?

I'm using Java with spark 1.6.2:

Map<String, String> options = new HashMap<String, String>() {{
  put("url", "jdbc:orient:plocal:./target/databases/sparkTest");
  put("dbtable", "Item");
}};

SQLContext sqlCtx = new SQLContext(ctx);

DataFrame jdbcDF = sqlCtx.read().format("jdbc").options(options).load();


I found that someone has the same problem with SAS JDBC.
As a workaround I will implement a query cleaner inside the driver,
but an option to configure the quoting char would be better.

Regards,
RF

-- 
Roberto Franchini
"The impossible is inevitable"
https://github.com/robfrank/ https://twitter.com/robfrankie
hangout:ro.franchini skype:ro.franchini

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message