spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <so...@cloudera.com>
Subject Re: Running Google Dataflow on Spark
Date Wed, 02 Nov 2016 12:07:42 GMT
This is a Dataflow / Beam question, not a Spark question per se.

On Wed, Nov 2, 2016 at 11:48 AM Ashutosh Kumar <kmr.ashutosh16@gmail.com>
wrote:

> I am trying to run Google Dataflow code on Spark. It works fine as google
> dataflow on google cloud platform. But while running on Spark I am getting
> following error
>
> 16/11/02 11:14:32 INFO com.cloudera.dataflow.spark.SparkPipelineRunner:
> Evaluating ParDo(GroupByKeyHashAndSortByKeyAndWindow)
> Exception in thread "main" java.lang.RuntimeException:
> java.lang.IllegalStateException: No TransformEvaluator registered for class
> com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner$Grou
> pByKeyAndSortValuesOnly
>         at
> com.cloudera.dataflow.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:124)
>
> Thanks
> Ashutosh
>

Mime
View raw message