flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elias Levy (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-11794) Allow compression of row format files created by StreamingFileSink
Date Sat, 02 Mar 2019 07:29:00 GMT
Elias Levy created FLINK-11794:
----------------------------------

             Summary: Allow compression of row format files created by StreamingFileSink
                 Key: FLINK-11794
                 URL: https://issues.apache.org/jira/browse/FLINK-11794
             Project: Flink
          Issue Type: Improvement
          Components: Connectors / FileSystem
    Affects Versions: 1.7.2
            Reporter: Elias Levy


Currently, there is no mechanism to compress files created using a StreamingFileSink.  This
is highly desirable when output is a text based row format such as JSON.

Possible alternatives are the introduction of a callback that gets passed the local file before
it is uploaded to the DFS, so that it could be compressed; or a factory method could be used
that returns an OutputStream, such as GZIPOutputStream, that compresses a passed in output stream
that could be then used by the Encoder.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message