spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Praveen Tallapudi (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-19687) SPARK supports for Postgres JSONB to store JSON data, if yes, kindly please help us with any examples.
Date Tue, 21 Feb 2017 23:05:44 GMT
Praveen Tallapudi created SPARK-19687:
-----------------------------------------

             Summary: SPARK supports for Postgres JSONB to store JSON data, if yes, kindly
please help us with any examples.
                 Key: SPARK-19687
                 URL: https://issues.apache.org/jira/browse/SPARK-19687
             Project: Spark
          Issue Type: Question
          Components: Spark Core
    Affects Versions: 2.1.1
            Reporter: Praveen Tallapudi


Dear Team,
I am little new to Scala development and trying to find the solution for the below. Please
forgive me if this is not the correct place to post this question.

I am trying to insert data from a data frame into postgres table.
 
Dataframe Schema:
root
|-- ID: string (nullable = true)
|-- evtInfo: struct (nullable = true)
|    |-- @date: string (nullable = true)
|    |-- @time: string (nullable = true)
|    |-- @timeID: string (nullable = true)
|    |-- TranCode: string (nullable = true)
|    |-- custName: string (nullable = true)
|    |-- evtInfo: array (nullable = true)
|    |    |-- element: string (containsNull = true)
|    |-- Type: string (nullable = true)
|    |-- opID: string (nullable = true)
|    |-- tracNbr: string (nullable = true)
 
 
DataBase Table Schema:
CREATE TABLE public.test
(
   id bigint NOT NULL,   
   evtInfo jsonb NOT NULL,
   evt_val bigint NOT NULL
)
 
When I use dataFrame_toSave.write.mode(SaveMode.Append).jdbc(dbUrl, "public.test", dbPropForDFtoSave)
to save the data, I am seeing the below error.
 
Exception in thread "main" java.lang.IllegalArgumentException: Can't get JDBC type for struct<@dateEvt:string,@timeEvt:string,@timeID:string,CICSTranCode:string,custName:string,evtInfo:array<string>,evtType:string,operID:string,trackingNbr:string>
 
Can you please suggest the best approach to save the data frame into the posgres JSONB table.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message