spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ranju goel <goel.ra...@gmail.com>
Subject RepartitionByCassandraReplica API Support on K8s
Date Fri, 04 Jun 2021 08:19:00 GMT
Hi All,

I am running Spark 3.0.1 on Kubernetes where Spark fetching data from
Cassandra and stores it in a JavaRDD.

My Question is Does RDD JavaFunctions *repartitionByCassandraReplica *works
on Kubernetes environment. I can get the result if I am using it in case of
Spark Stand Alone on Virtualized Environment but as if I use the same
API (*repartitionByCassandraReplica
* ) on Kubernetes , spark RDD return as empty.

*API :*
CassandraJavaUtil.javaFunctions(theJavaRDD).repartitionByCassandraReplica(keyspaceName,
tableName, partitionsPerHost, partitionkeyMapper, rowWriterFactory).

Please suggest Can Spark Data Locality awareness can be achieved in
Kubernetes as well as availability of this feature directly
impacts performance.

Regards
User

Mime
View raw message