beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [beam] branch asf-site updated: Publishing website 2018/11/01 23:42:17 at commit ccb0fcb
Date Thu, 01 Nov 2018 23:42:22 GMT
This is an automated email from the ASF dual-hosted git repository.

git-site-role pushed a commit to branch asf-site
in repository

The following commit(s) were added to refs/heads/asf-site by this push:
     new 67f4781  Publishing website 2018/11/01 23:42:17 at commit ccb0fcb
67f4781 is described below

commit 67f478124753faff3cf8959706bc28e972ae23e8
Author: jenkins <>
AuthorDate: Thu Nov 1 23:42:18 2018 +0000

    Publishing website 2018/11/01 23:42:17 at commit ccb0fcb
 .../generated-content/roadmap/go-sdk/index.html    | 78 +++++++++++++++++++---
 1 file changed, 68 insertions(+), 10 deletions(-)

diff --git a/website/generated-content/roadmap/go-sdk/index.html b/website/generated-content/roadmap/go-sdk/index.html
index 88ccbda..d3780d0 100644
--- a/website/generated-content/roadmap/go-sdk/index.html
+++ b/website/generated-content/roadmap/go-sdk/index.html
@@ -191,6 +191,12 @@
+<ul class="nav">
+  <li><a href="#usability">Usability</a></li>
+  <li><a href="#performance">Performance</a></li>
+  <li><a href="#integrating-with-the-go-ecosystem">Integrating with the Go ecosystem</a></li>
+  <li><a href="#implement-portability-features">Implement Portability features</a></li>
@@ -212,19 +218,71 @@ limitations under the License.
 <h1 id="go-sdk-roadmap">Go SDK Roadmap</h1>
-<p>This roadmap is in progress. In the meantime, here are available resources:</p>
+<p>The Go SDK is currently experimental. As the first purely portable Beam SDK, the
Go SDK is constrained
+by the status of the <a href="">Beam Portabillity
Framework</a> and the existence of
+portable runners.</p>
+<p>With that in mind present work on the Go SDK has the following goals:</p>
+<h2 id="usability">Usability</h2>
+<p>The Go SDK should not block users from writing the pipelines they need to run.</p>
-  <li>
-    <table>
-      <tbody>
-        <tr>
-          <td>JIRA: <a href="">sdk-go</a>
/ <a href="">BEAM-2083</a></td>
-        </tr>
-      </tbody>
-    </table>
+  <li>Support large elements <a href="">BEAM-4124</a></li>
+  <li>Coder Registry to support complex user elements <a href="">BEAM-3306</a></li>
+<h2 id="performance">Performance</h2>
+<p>Since the Go SDK has no official runner support at this time, this gives us the
chance to work on the performance
+of the SDK harness. When portable runners are ready, the Go SDK should be ready as well.</p>
+  <li>Reducing framework overhead on the SDK Harness side <a href="">BEAM-4726</a></li>
+  <li>Tooling to enable and verify pipeline performance <a href="">BEAM-3612</a></li>
+  <li>Measure the performance of the SDK at scale.</li>
+<h2 id="integrating-with-the-go-ecosystem">Integrating with the Go ecosystem</h2>
+<p>The authors of the Go SDK intended to keep the parts of the language we love, and
intend for
+user pipelines to be as close to ordinary Go as possible, minimizing framework boiler plate
+  <li>Migrate to a vanity URL path <a href="">BEAM-4115</a></li>
+  <li>Package versioning support <a href="">BEAM-5379</a></li>
+<h2 id="implement-portability-features">Implement Portability features</h2>
+<p>The following are dependant on the features being implemented in the Portability
+  <li>Splitable DoFns (SDFs) for Scalable IO <a href="">BEAM-3301</a>
+    <ul>
+      <li>This is the primary blocker for the Go SDK to scale to ingest large numbers
of elements in both batch and streaming.</li>
+    </ul>
-  <li>Contact: <a href="">Henning Rohde</a></li>
+  <li>Triggers and Timers <a href="">BEAM-3304</a></li>
+  <li>Advanced WindowFns
+    <ul>
+      <li>Session windows <a href="">BEAM-4152</a></li>
+      <li>Custom WindowFns (dependant on streaming Splitable DoFn support) <a href="">BEAM-2939</a></li>
+    </ul>
+  </li>
+<p>Without SDFs, IOs are always constrained to work on a single machine prior to a
sharding operation (like CoGroupByKey),
+which makes scalable IOs difficult to impossible to write.</p>
+<p>Otherwise, improving examples and documentation for devs and users alike is ongoing.

+Contributions are welcome. Please contact the <a href="">dev
+for assistance in finding a place to help out.</p>
+  <li>JIRA: <a href="">sdk-go</a>
/ <a href="">BEAM-2083</a></li>
+  <li>Contact: Robert Burke (@lostluck) <a href=",">Email</a></li>

View raw message