beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [beam] branch master updated: Add an initial Go SDK Roadmap (#6901)
Date Thu, 01 Nov 2018 23:41:40 GMT
This is an automated email from the ASF dual-hosted git repository.

altay pushed a commit to branch master
in repository

The following commit(s) were added to refs/heads/master by this push:
     new ccb0fcb  Add an initial Go SDK Roadmap (#6901)
ccb0fcb is described below

commit ccb0fcbb469972c3e195b7cdde29ac6c35d6e459
Author: Robert Burke <>
AuthorDate: Thu Nov 1 16:41:34 2018 -0700

    Add an initial Go SDK Roadmap (#6901)
    * Add an initial Go SDK Roadmap
 website/src/roadmap/ | 52 ++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 49 insertions(+), 3 deletions(-)

diff --git a/website/src/roadmap/ b/website/src/roadmap/
index 0faef82..c87b1ed 100644
--- a/website/src/roadmap/
+++ b/website/src/roadmap/
@@ -20,7 +20,53 @@ limitations under the License.
 # Go SDK Roadmap
-This roadmap is in progress. In the meantime, here are available resources:
+The Go SDK is currently experimental. As the first purely portable Beam SDK, the Go SDK is
+by the status of the [Beam Portabillity Framework](
and the existence of
+portable runners.
- - JIRA: [sdk-go](
/ [BEAM-2083]( |
- - Contact: [Henning Rohde](
+With that in mind present work on the Go SDK has the following goals:
+## Usability
+The Go SDK should not block users from writing the pipelines they need to run.
+* Support large elements [BEAM-4124](
+* Coder Registry to support complex user elements [BEAM-3306](
+## Performance
+Since the Go SDK has no official runner support at this time, this gives us the chance to
work on the performance
+of the SDK harness. When portable runners are ready, the Go SDK should be ready as well.
+* Reducing framework overhead on the SDK Harness side [BEAM-4726](
+* Tooling to enable and verify pipeline performance [BEAM-3612](
+* Measure the performance of the SDK at scale.
+## Integrating with the Go ecosystem
+The authors of the Go SDK intended to keep the parts of the language we love, and intend
+user pipelines to be as close to ordinary Go as possible, minimizing framework boiler plate
+* Migrate to a vanity URL path [BEAM-4115](
+* Package versioning support [BEAM-5379](
+## Implement Portability features
+The following are dependant on the features being implemented in the Portability Framework.
+* Splitable DoFns (SDFs) for Scalable IO [BEAM-3301](
+  * This is the primary blocker for the Go SDK to scale to ingest large numbers of elements
in both batch and streaming.
+* Triggers and Timers [BEAM-3304](
+* Advanced WindowFns
+   * Session windows [BEAM-4152](
+   * Custom WindowFns (dependant on streaming Splitable DoFn support) [BEAM-2939](
+Without SDFs, IOs are always constrained to work on a single machine prior to a sharding
operation (like CoGroupByKey),
+which makes scalable IOs difficult to impossible to write.
+Otherwise, improving examples and documentation for devs and users alike is ongoing. 
+Contributions are welcome. Please contact the [dev list](
+for assistance in finding a place to help out.
+ - JIRA: [sdk-go](
/ [BEAM-2083](
+ - Contact: Robert Burke (@lostluck) [Email](,

View raw message