beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From git-site-r...@apache.org
Subject [beam] branch asf-site updated: Publishing website 2021/06/11 06:01:30 at commit f4430e6
Date Fri, 11 Jun 2021 06:02:19 GMT
This is an automated email from the ASF dual-hosted git repository.

git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/asf-site by this push:
     new 4c9c273  Publishing website 2021/06/11 06:01:30 at commit f4430e6
4c9c273 is described below

commit 4c9c273b76ff908987f78bf5d90a638e49ba0afd
Author: jenkins <builds@apache.org>
AuthorDate: Fri Jun 11 06:01:31 2021 +0000

    Publishing website 2021/06/11 06:01:30 at commit f4430e6
---
 .../documentation/basics/index.html                | 18 +++++++---------
 website/generated-content/documentation/index.xml  | 24 ++++++----------------
 website/generated-content/sitemap.xml              |  2 +-
 3 files changed, 14 insertions(+), 30 deletions(-)

diff --git a/website/generated-content/documentation/basics/index.html b/website/generated-content/documentation/basics/index.html
index ee848cb..a1d78b7 100644
--- a/website/generated-content/documentation/basics/index.html
+++ b/website/generated-content/documentation/basics/index.html
@@ -31,15 +31,11 @@ processing engine.</li></ul><p>These concepts may be
very similar to your proces
 Beam&rsquo;s design is for cross-language operation and reusable libraries of
 transforms, there are some special features worth highlighting.</p><h3 id=pipeline>Pipeline</h3><p>A
pipeline in Beam is a graph of PTransforms operating on PCollections. A
 pipeline is constructed by a user in their SDK of choice, and makes its way to
-your runner either via the SDK directly or via the Runner API&rsquo;s (forthcoming)
-RPC interfaces.</p><h3 id=ptransforms>PTransforms</h3><p>In Beam,
a PTransform can be one of the five primitives or it can be a
-composite transform encapsulating a subgraph. The primitives are:</p><ul><li><a
href=#implementing-the-read-primitive><em>Read</em></a> - parallel connectors
to external
-systems</li><li><a href=#implementing-the-pardo-primitive><em>ParDo</em></a>
- per element processing</li><li><a href=#implementing-the-groupbykey-and-window-primitive><em>GroupByKey</em></a>
-
-aggregating elements per key and window</li><li><a href=#implementing-the-flatten-primitive><em>Flatten</em></a>
- union of PCollections</li><li><a href=#implementing-the-window-primitive><em>Window</em></a>
- set the windowing strategy
-for a PCollection</li></ul><p>When implementing a runner, these are the
operations you need to implement.
-Composite transforms may or may not be important to your runner. If you expose
-a UI, maintaining some of the composite structure will make the pipeline easier
-for a user to understand. But the result of processing is not changed.</p><h3 id=pcollections>PCollections</h3><p>A
PCollection is an unordered bag of elements. Your runner will be responsible
+your runner either via the SDK directly or via the Runner API&rsquo;s
+RPC interfaces.</p><h3 id=ptransforms>PTransforms</h3><p>A <code>PTransform</code>
represents a data processing operation, or a step,
+in your pipeline. A <code>PTransform</code> can be applied to one or more
+<code>PCollection</code> objects as input which performs some processing on the
elements of that
+<code>PCollection</code> and produces zero or more output <code>PCollection</code>
objects.</p><h3 id=pcollections>PCollections</h3><p>A PCollection
is an unordered bag of elements. Your runner will be responsible
 for storing these elements. There are some major aspects of a PCollection to
 note:</p><h4 id=bounded-vs-unbounded>Bounded vs Unbounded</h4><p>A
PCollection may be bounded or unbounded.</p><ul><li><em>Bounded</em>
- it is finite and you know it, as in batch use cases</li><li><em>Unbounded</em>
- it may be never end, you don&rsquo;t know, as in streaming use cases</li></ul><p>These
derive from the intuitions of batch and stream processing, but the two
 are unified in Beam and bounded and unbounded PCollections can coexist in the
@@ -87,7 +83,7 @@ in side inputs)</li><li><em>WindowMappingFn</em>
- maps one element&rsquo;s wind
 bounds on how far in the past the result window will be (used in side
 inputs)</li><li><em>CombineFn</em> - associative and commutative
aggregation (used in Combine and
 state)</li><li><em>Coder</em> - encodes user data; some coders have
standard formats and are not really UDFs</li></ul><p>The various types of
user-defined functions will be described further alongside
-the primitives that use them.</p><h3 id=runner>Runner</h3><p>The
term &ldquo;runner&rdquo; is used for a couple of things. It generally refers to the
+the <a href=#ptransforms><em>PTransforms</em></a> that use them.</p><h3
id=runner>Runner</h3><p>The term &ldquo;runner&rdquo; is used for a
couple of things. It generally refers to the
 software that takes a Beam pipeline and executes it somehow. Often, this is the
 translation code that you write. It usually also includes some customized
 operators for your data processing engine, and is sometimes used to refer to
@@ -96,7 +92,7 @@ use code font for proper nouns in our APIs, whether or not the identifiers
 match across all SDKs.</p><p>The <code>run(Pipeline)</code> method
should be asynchronous and results in a
 PipelineResult which generally will be a job descriptor for your data
 processing engine, providing methods for checking its status, canceling it, and
-waiting for it to terminate.</p><div class=feedback><p class=update>Last
updated on 2021/02/05</p><h3>Have you found everything you were looking for?</h3><p
class=description>Was it all useful and clear? Is there anything that you would like to
change? Let us know!</p><button class=load-button><a href="mailto:dev@beam.apache.org?subject=Beam
Website Feedback">SEND FEEDBACK</a></button></div></div></div><footer
class=footer><div class=footer__contained><div class=footer__cols><div class="
[...]
+waiting for it to terminate.</p><div class=feedback><p class=update>Last
updated on 2021/06/03</p><h3>Have you found everything you were looking for?</h3><p
class=description>Was it all useful and clear? Is there anything that you would like to
change? Let us know!</p><button class=load-button><a href="mailto:dev@beam.apache.org?subject=Beam
Website Feedback">SEND FEEDBACK</a></button></div></div></div><footer
class=footer><div class=footer__contained><div class=footer__cols><div class="
[...]
 <a href=http://www.apache.org>The Apache Software Foundation</a>
 | <a href=/privacy_policy>Privacy Policy</a>
 | <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam,
the Beam logo, and the Apache feather logo are either registered trademarks or trademarks
of The Apache Software Foundation. All other products or name brands are trademarks of their
respective holders, including The Apache Software Foundation.</div></div></div></div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/documentation/index.xml b/website/generated-content/documentation/index.xml
index 03f52d5..8c869ad 100644
--- a/website/generated-content/documentation/index.xml
+++ b/website/generated-content/documentation/index.xml
@@ -3204,25 +3204,13 @@ transforms, there are some special features worth highlighting.&lt;/p>
 &lt;h3 id="pipeline">Pipeline&lt;/h3>
 &lt;p>A pipeline in Beam is a graph of PTransforms operating on PCollections. A
 pipeline is constructed by a user in their SDK of choice, and makes its way to
-your runner either via the SDK directly or via the Runner API&amp;rsquo;s (forthcoming)
+your runner either via the SDK directly or via the Runner API&amp;rsquo;s
 RPC interfaces.&lt;/p>
 &lt;h3 id="ptransforms">PTransforms&lt;/h3>
-&lt;p>In Beam, a PTransform can be one of the five primitives or it can be a
-composite transform encapsulating a subgraph. The primitives are:&lt;/p>
-&lt;ul>
-&lt;li>&lt;a href="#implementing-the-read-primitive">&lt;em>Read&lt;/em>&lt;/a>
- parallel connectors to external
-systems&lt;/li>
-&lt;li>&lt;a href="#implementing-the-pardo-primitive">&lt;em>ParDo&lt;/em>&lt;/a>
- per element processing&lt;/li>
-&lt;li>&lt;a href="#implementing-the-groupbykey-and-window-primitive">&lt;em>GroupByKey&lt;/em>&lt;/a>
-
-aggregating elements per key and window&lt;/li>
-&lt;li>&lt;a href="#implementing-the-flatten-primitive">&lt;em>Flatten&lt;/em>&lt;/a>
- union of PCollections&lt;/li>
-&lt;li>&lt;a href="#implementing-the-window-primitive">&lt;em>Window&lt;/em>&lt;/a>
- set the windowing strategy
-for a PCollection&lt;/li>
-&lt;/ul>
-&lt;p>When implementing a runner, these are the operations you need to implement.
-Composite transforms may or may not be important to your runner. If you expose
-a UI, maintaining some of the composite structure will make the pipeline easier
-for a user to understand. But the result of processing is not changed.&lt;/p>
+&lt;p>A &lt;code>PTransform&lt;/code> represents a data processing operation,
or a step,
+in your pipeline. A &lt;code>PTransform&lt;/code> can be applied to one or
more
+&lt;code>PCollection&lt;/code> objects as input which performs some processing
on the elements of that
+&lt;code>PCollection&lt;/code> and produces zero or more output &lt;code>PCollection&lt;/code>
objects.&lt;/p>
 &lt;h3 id="pcollections">PCollections&lt;/h3>
 &lt;p>A PCollection is an unordered bag of elements. Your runner will be responsible
 for storing these elements. There are some major aspects of a PCollection to
@@ -3309,7 +3297,7 @@ state)&lt;/li>
 &lt;li>&lt;em>Coder&lt;/em> - encodes user data; some coders have standard
formats and are not really UDFs&lt;/li>
 &lt;/ul>
 &lt;p>The various types of user-defined functions will be described further alongside
-the primitives that use them.&lt;/p>
+the &lt;a href="#ptransforms">&lt;em>PTransforms&lt;/em>&lt;/a>
that use them.&lt;/p>
 &lt;h3 id="runner">Runner&lt;/h3>
 &lt;p>The term &amp;ldquo;runner&amp;rdquo; is used for a couple of things.
It generally refers to the
 software that takes a Beam pipeline and executes it somehow. Often, this is the
diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml
index 0131fdd..6d3818b 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.30.0/</loc><lastmod>2021-05-26T19:50:36-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2021-06-09T10:56:20-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2021-06-09T10:56:20-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2021-06-09T10:56:20-07:00</lastmod></url><url><loc>/blog/v
[...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.30.0/</loc><lastmod>2021-05-26T19:50:36-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2021-06-09T10:56:20-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2021-06-09T10:56:20-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2021-06-09T10:56:20-07:00</lastmod></url><url><loc>/blog/v
[...]
\ No newline at end of file

Mime
View raw message