beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Baptiste Onofré (JIRA) <>
Subject [jira] [Commented] (BEAM-1581) JSON source and sink
Date Tue, 09 May 2017 13:07:04 GMT


Jean-Baptiste Onofré commented on BEAM-1581:

I think same proposal/idea as in BEAM-2228 can apply to "JsonIO/JacksonIO".

> JSON source and sink
> --------------------
>                 Key: BEAM-1581
>                 URL:
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-java-extensions
>            Reporter: Aviem Zur
> JSON source and sink to read/write JSON files.
> Similarly to {{XmlSource}}/{{XmlSink}}, these be a {{JsonSource}}/{{JonSink}} which are
a {{FileBaseSource}}/{{FileBasedSink}}.
> Consider using methods/code (or refactor these) found in {{AsJsons}} and {{ParseJsons}}
> The {{PCollection}} of objects the user passes to the transform should be embedded in
a valid JSON file
> The most common pattern for this is a large object with an array member which holds all
the data objects and other members for metadata.
> Examples of public JSON APIs:
> Another pattern used is a file which is simply a JSON array of objects.

This message was sent by Atlassian JIRA

View raw message