spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix Cheung <>
Subject Re: Fwd: Writing to jdbc database from SparkR (1.5.2)
Date Sun, 07 Feb 2016 01:54:21 GMT
Unfortunately I couldn't find a simple workaround. It seems to be an issue with
that does not work with jdbc source/format
For instance, this does not work in Scala eitherdf1.write.format("jdbc").mode("overwrite").option("url",
For Spark 1.5.x, it seems the best option would be to write a JVM wrapper and call it from

From: Andrew Holway <>
Sent: Saturday, February 6, 2016 11:22 AM
Subject: Fwd: Writing to jdbc database from SparkR (1.5.2)
To:  <>

          I have a thread on but I think this might require developer
             I'm reading data from a database: This is working well.     

> df <- read.df(sqlContext, source="jdbc", url="jdbc:mysql://")
                          When I try and write something back to the DB I see this following

> write.df(fooframe, path="NULL", source="jdbc", url="jdbc:mysql://",
dbtable="db.table", mode="append")       


16/02/06 19:05:43 ERROR RBackendHandler: save on 2 failed        

Error in invokeJava(isStatic = FALSE, objId$id, methodName, ...) :  

  java.lang.RuntimeException: org.apache.spark.sql.execution.datasources.jdbc.DefaultSource
does not allow create table as select. 

 at scala.sys.package$.error(package.scala:27) 

 at org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:200)



 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 

 at sun.reflect.NativeMethodAccessorImpl.invoke( 

 at sun.reflect.DelegatingMethodAccessorImpl.invoke(

 at java.lang.reflect.Method.invoke( 

 at org.apache.spark.api.r.RBackendHandler.handleMethodCall(RBackendHandler.scala:132) 

 at org.apache.spark.api.r.RBackendHandler.channelRead0(RBackendHandler.scala:79) 

 at org.apache.spark.api.r.RBackendHandler.channelRead0(RBackendHandler.scala:38) 


Any ideas on a workaround?



View raw message