spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Boesch <java...@gmail.com>
Subject Re: Spark SQL 1.5.2 missing JDBC driver for PostgreSQL?
Date Wed, 23 Dec 2015 02:40:55 GMT
HI Benjamin,  yes by adding to the thrift server then the create table
would work.  But querying is performed by the workers: so you need to add
to the classpath of all nodes for reads to work.

2015-12-22 18:35 GMT-08:00 Benjamin Kim <bbuild11@gmail.com>:

> Hi Stephen,
>
> I forgot to mention that I added these lines below to the
> spark-default.conf on the node with Spark SQL Thrift JDBC/ODBC Server
> running on it. Then, I restarted it.
>
> spark.driver.extraClassPath=/usr/share/java/postgresql-9.3-1104.jdbc41.jar
>
> spark.executor.extraClassPath=/usr/share/java/postgresql-9.3-1104.jdbc41.jar
>
> I read in another thread that this would work. I was able to create the
> table and could see it in my SHOW TABLES list. But, when I try to query the
> table, I get the same error. It looks like I’m getting close.
>
> Are there any other things that I have to do that you can think of?
>
> Thanks,
> Ben
>
>
> On Dec 22, 2015, at 6:25 PM, Stephen Boesch <javadba@gmail.com> wrote:
>
> The postgres jdbc driver needs to be added to the  classpath of your spark
> workers.  You can do a search for how to do that (multiple ways).
>
> 2015-12-22 17:22 GMT-08:00 b2k70 <bbuild11@gmail.com>:
>
>> I see in the Spark SQL documentation that a temporary table can be created
>> directly onto a remote PostgreSQL table.
>>
>> CREATE TEMPORARY TABLE <table_name>
>> USING org.apache.spark.sql.jdbc
>> OPTIONS (
>> url "jdbc:postgresql://<PostgreSQL_Hostname_IP>/<database_name>",
>> dbtable "impressions"
>> );
>> When I run this against our PostgreSQL server, I get the following error.
>>
>> Error: java.sql.SQLException: No suitable driver found for
>> jdbc:postgresql://<PostgreSQL_Hostname_IP>/<database_name> (state=,code=0)
>>
>> Can someone help me understand why this is?
>>
>> Thanks, Ben
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-1-5-2-missing-JDBC-driver-for-PostgreSQL-tp25773.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com
>> <http://nabble.com>.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>>
>
>

Mime
View raw message