beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chaoran Yu (JIRA)" <>
Subject [jira] [Commented] (BEAM-302) Add Scio Scala DSL to Beam
Date Wed, 03 May 2017 00:35:04 GMT


Chaoran Yu commented on BEAM-302:

After including SparkRunner dependency, I got the following exception "java.lang.NoClassDefFoundError:
org/apache/spark/streaming/api/java/JavaStreamingContextFactory", which is a class that has
ceased to exist for Spark 2.0+. I'm sure by digging deeper, the word count example can be
run with SparkRunner. But my idea is that currently it's still not as simple as it should
be. Spark runner, or basically most other runners supported by Beam, should receive the same
support from scio as is the case for GCD runner. Is this an aspect where Spotify would like
to see contributions from the community?

> Add Scio Scala DSL to Beam
> --------------------------
>                 Key: BEAM-302
>                 URL:
>             Project: Beam
>          Issue Type: Wish
>          Components: sdk-ideas
>            Reporter: Jean-Baptiste Onofré

This message was sent by Atlassian JIRA

View raw message