spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From KhajaAsmath Mohammed <mdkhajaasm...@gmail.com>
Subject Spark submit hbase issue
Date Wed, 14 Apr 2021 21:38:59 GMT
Hi,

Spark submit is connecting to local host instead of zookeeper mentioned in hbase-site.xml.
This same program works in ide which gets connected to hbase-site.xml. What am I missing in
spark submit?
> 
> 
> spark-submit --driver-class-path  C:\Users\mdkha\bitbucket\clx-spark-scripts\src\test\resources\hbase-site.xml
--files C:\Users\mdkha\bitbucket\clx-spark-scripts\src\test\resources\hbase-site.xml  --conf
"spark.driver.extraLibraryPath=C:\Users\mdkha\bitbucket\clx-spark-scripts\src\test\resources\hbase-site.xml"
--executor-memory 4g --class com.drivers.HBASEExportToS3 C:\Users\mdkha\bitbucket\clx-spark-scripts\target\clx-spark-scripts.jar
-c C:\Users\mdkha\bitbucket\clx-spark-scripts\src\test\resources\job.properties 
> 
> 
> 21/04/14 16:13:36 WARN ClientCnxn: Session 0x0 for server null, unexpected error, closing
socket connection and attempting reconnect
> java.net.ConnectException: Connection refused: no further information
>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:715)
>         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
>         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
> 21/04/14 16:13:37 WARN ReadOnlyZKClient: 0x2d73767e to localhost:2181 failed for get
of /hbase/hbaseid, code = CONNECTIONLOSS, retries = 1

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message