ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Praneeth Ramesh (Jira)" <j...@apache.org>
Subject [jira] [Created] (IGNITE-12519) Spark SQL not working with NON upper case column names
Date Wed, 01 Jan 2020 00:28:00 GMT
Praneeth Ramesh created IGNITE-12519:

             Summary: Spark SQL not working with NON upper case column names 
                 Key: IGNITE-12519
                 URL: https://issues.apache.org/jira/browse/IGNITE-12519
             Project: Ignite
          Issue Type: Bug
          Components: spark
    Affects Versions: 2.7.6
         Environment: 1) Spark 2.3.0 (Tried on Mesos Master and Local Master)

2) Ignite 2.7.6 (10 Nodes Cluster on Kubernetes)

3) Spark Ignite 2.7.6
            Reporter: Praneeth Ramesh

I created a simple table as below.
  "accCol1" VARCHAR,
  "accCol2" INT,
  "accCol3" VARCHAR,
  "accCol4" BOOLEAN
And trying to read the data from table from Ignite Spark as below.

val igniteDF = spark.read
 .option(OPTION_TABLE, "acc")
 .option(OPTION_CONFIG_FILE, "example-config.xml")
igniteDF.show(100, false)

But I see an exception as below.
Caused by: org.h2.jdbc.JdbcSQLException: Column "ACCCOL1" not found; SQL statement:
SELECT accCol4, CAST(accCol1 AS VARCHAR) AS accCol1, accCol2, CAST(accCol3 AS VARCHAR) AS
accCol3, accId FROM ACC LIMIT 21 [42122-197]
 at org.h2.message.DbException.getJdbcSQLException(DbException.java:357)
 at org.h2.message.DbException.get(DbException.java:179)
 at org.h2.message.DbException.get(DbException.java:155)
 at org.h2.expression.ExpressionColumn.optimize(ExpressionColumn.java:150)
 at org.h2.command.dml.Select.prepare(Select.java:858)
 at org.h2.command.Parser.prepareCommand(Parser.java:283)
 at org.h2.engine.Session.prepareLocal(Session.java:611)
 at org.h2.engine.Session.prepareCommand(Session.java:549)
 at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1247)
 at org.h2.jdbc.JdbcPreparedStatement.<init>(JdbcPreparedStatement.java:76)
 at org.h2.jdbc.JdbcConnection.prepareStatement(JdbcConnection.java:694)
 at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.prepare0(IgniteH2Indexing.java:539)
 at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.prepareStatement(IgniteH2Indexing.java:509)
 at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.prepareStatement(IgniteH2Indexing.java:476){code}

When I try naming the TABLE cols with UPPER CASE everything works fine. But when I use the
quotes in the Column Names to preserve the case, then it breaks with the exception.

>From exception I can see query built is still having the UPPER case column name ACCCOL1
instead of the camel case column names.

Is there any workaround for this.


This message was sent by Atlassian Jira

View raw message