kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Henke <ghe...@cloudera.com>
Subject Re: Kudu Reader is not working with Java
Date Mon, 22 Apr 2019 15:37:44 GMT
The ability to specify the format as simply "kudu" was added in 1.9.0. See
the release notes here:
https://kudu.apache.org/releases/1.9.0/docs/release_notes.html#rn_1.9.0_improvements

I would recommend upgrading to 1.9.0 or using the fully qualified name
"org.apache.kudu.spark.kudu"

Thanks,
Grant

On Mon, Apr 22, 2019 at 10:23 AM Ankit Kumar Mishra <Ankit.Mishra@wu.com>
wrote:

> Hello Team,
>
>
>
> My Kudu Reader Java implementation is getting failed with exception - “java.lang.ClassNotFoundException:
> Failed to find data source: kudu. Please find packages at
> http://spark.apache.org/third-party-projects.html”
>
>
>
> *Java Implementation*
>
>
>
> String kuduTableName = *"impala::" *+ *kuduConfig*.getDatabaseName() + *"."
> *+ *kuduConfig*.getInTableName();
>
> Map<String,String> kuduWriterMap = *new *HashMap<String, String>();
> kuduWriterMap.put(*"kudu.master"*,*kuduConfig*.getMasterAddress());
> kuduWriterMap.put(*"kudu.table"*,kuduTableName);
> kuduWriterMap.put(*"kudu.socketReadTimeoutMs"*,*"720000"*);
>
> Dataset<Row> readerDS = *sparkSession*
> .read().options(kuduWriterMap).format(*"kudu"*).load();
>
>
>
> I have already defined below dependencies in my Pom.xml and jars are
> present in my External library.
>
>
>
> <*dependency*>
>     <*groupId*>org.apache.kudu</*groupId*>
>     <*artifactId*>kudu-client</*artifactId*>
>     <*version*>1.8.0</*version*>
>     <*scope*>test</*scope*>
> </*dependency*>
>
> <*dependency*>
>     <*groupId*>org.apache.kudu</*groupId*>
>     <*artifactId*>kudu-spark2_2.11</*artifactId*>
>     <*version*>1.8.0</*version*>
> </*dependency*>
>
> <*dependency*>
>     <*groupId*>org.apache.kudu</*groupId*>
>     <*artifactId*>kudu-spark2-tools_2.11</*artifactId*>
>     <*version*>1.8.0</*version*>
> </*dependency*>
>
> <*dependency*>
>     <*groupId*>org.apache.kudu</*groupId*>
>     <*artifactId*>kudu-client</*artifactId*>
>     <*version*>1.8.0</*version*>
>     <*type*>test-jar</*type*>
>     <*scope*>test</*scope*>
> </*dependency*>
>
>
>
> Please help.
>
>
>
> Regards,
>
> Ankit Mishra
>
>
> The information transmitted, including any content in this communication
> is confidential, is intended only for the use of the intended recipient and
> is the property of the Western Union Company or its affiliates and
> subsidiaries. If you are not the intended recipient, you are hereby
> notified that any use of the information contained in or transmitted with
> the communication or dissemination, distribution, or copying of this
> communication is strictly prohibited. If you have received this
> communication in error, please notify the Western Union sender immediately
> by replying to this message and delete the original message
>


-- 
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke

Mime
View raw message