spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deepa Jayaveer <deepa.jayav...@tcs.com>
Subject Re: reg JDBCRDD code
Date Wed, 10 Dec 2014 07:15:59 GMT
Thanks Akhil but it is expecting Function1 instead of Function .. I tried 
out writing a new class by implementing Function1 but
got an error . can you please help us to get it resolved

JDBCRDD is created as
JdbcRDD rdd = new JdbcRDD(sc, getConnection, sql, 0, 0, 1,
                getResultset, ClassTag$.MODULE$.apply(String.class));


overridden  'apply' method in Function1 
public String apply(ResultSet arg0) {

             String ss = null;

            try {
                ss = (String) ((java.sql.ResultSet) arg0).getString(1);
            } catch (SQLException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }

            System.out.println(ss);

            return ss;
            // TODO Auto-generated method stub
        }

Error log
Exception in thread "main" org.apache.spark.SparkException: Job aborted 
due to stage failure: Task 0.0:0 failed 1 times, most recent failure: 
Exception failure in TID 0 on host localhost: java.sql.SQLException: 
Parameter index out of range (1 > number of parameters, which is 0).
        com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1073)
        com.mysql.jdbc.SQLError.createSQLException(SQLError.java:987)
        com.mysql.jdbc.SQLError.createSQLException(SQLError.java:982)
        com.mysql.jdbc.SQLError.createSQLException(SQLError.java:927)
        com.mysql.jdbc.PreparedStatement.checkBounds(
PreparedStatement.java:3709)
        com.mysql.jdbc.PreparedStatement.setInternal(
PreparedStatement.java:3693)
        com.mysql.jdbc.PreparedStatement.setInternal(
PreparedStatement.java:3735)
        com.mysql.jdbc.PreparedStatement.setLong(
PreparedStatement.java:3751)

Thanks 
Deepa



From:   Akhil Das <akhil@sigmoidanalytics.com>
To:     Deepa Jayaveer <deepa.jayaveer@tcs.com>
Cc:     "user@spark.apache.org" <user@spark.apache.org>
Date:   12/09/2014 09:30 PM
Subject:        Re: reg JDBCRDD code



Hi Deepa,

In Scala, You will do something like 
https://gist.github.com/akhld/ccafb27f098163bea622 

With Java API's it will be something like 
https://gist.github.com/akhld/0d9299aafc981553bc34



Thanks
Best Regards

On Tue, Dec 9, 2014 at 6:39 PM, Deepa Jayaveer <deepa.jayaveer@tcs.com> 
wrote:
Hi All, 
am new to Spark.  I tried to connect to Mysql using Spark.  want to write 
a code in Java but 
getting runtime exception. I guess that the issue is with the function0 
and function1 objects being passed in JDBCRDD . 

I tried my level best and attached the code, can you please help us to fix 
the issue. 



Thanks 
Deepa
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain 
confidential or privileged information. If you are 
not the intended recipient, any dissemination, use, 
review, distribution, printing or copying of the 
information contained in this e-mail message 
and/or attachments to it are strictly prohibited. If 
you have received this communication in error, 
please notify us by reply e-mail or telephone and 
immediately and permanently delete the message 
and any attachments. Thank you


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message