spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roberto Franchini <>
Subject OrientDB through JDBC: query field names wrapped by double quote
Date Tue, 09 Aug 2016 12:36:07 GMT
Hi to all,
I'm the maintainer of the JDBC driver OrientDB.

We are trying to fetch data to Spark from an orientDB using theJDBC driver.

I'm facing some issues:

To gather metadata spark performs a "test query" of this form: select
* from TABLE_NAME whre 1=0
For this case, I write a workaround inside the driver, getting rid of
where 1=0 and replaging it with LIMIT 1.

After that query, it then performs a query with each field wrapped by
double quote:

SELECT "stringKey", "intKey" FROM Item

In orientDB's SQL dialect a double quote means a string value, so for
each record of the result set it will return stringkey and intKey as

row 1: stringKey:strinKey, intKey:intKey
row 2: stringKey:strinKey, intKey:intKey
row 3: stringKey:strinKey, intKey:intKey

Is there a  way to configure SqlContext to avoid the double quoting of
fields names?

I'm using Java with spark 1.6.2:

Map<String, String> options = new HashMap<String, String>() {{
  put("url", "jdbc:orient:plocal:./target/databases/sparkTest");
  put("dbtable", "Item");

SQLContext sqlCtx = new SQLContext(ctx);

DataFrame jdbcDF ="jdbc").options(options).load();

I found that someone has the same problem with SAS JDBC.
As a workaround I will implement a query cleaner inside the driver,
but an option to configure the quoting char would be better.


Roberto Franchini
"The impossible is inevitable"
hangout:ro.franchini skype:ro.franchini

To unsubscribe e-mail:

View raw message