flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4771) Compression for AvroOutputFormat
Date Mon, 10 Oct 2016 12:13:20 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15562132#comment-15562132
] 

ASF GitHub Bot commented on FLINK-4771:
---------------------------------------

Github user fhueske commented on a diff in the pull request:

    https://github.com/apache/flink/pull/2612#discussion_r82591316
  
    --- Diff: flink-batch-connectors/flink-avro/src/main/java/org/apache/flink/api/java/io/AvroOutputFormat.java
---
    @@ -36,6 +38,8 @@
     	private final Class<E> avroValueType;
     
     	private transient Schema userDefinedSchema = null;
    +
    +	private transient CodecFactory codecFactory = null;
    --- End diff --
    
    Flink uses Java Serialization to ship code from the client to the cluster. `CodecFactory`
does not implement `Serializable`. Therefore, the `codecFactory` field must be declared as
`transient`. However, the `writeObject` and `readObject` methods must be adapted to manually
serialize / deserialize the information. Otherwise, the `CodecFactory` is lost when the object
is deserialized at the client.


> Compression for AvroOutputFormat
> --------------------------------
>
>                 Key: FLINK-4771
>                 URL: https://issues.apache.org/jira/browse/FLINK-4771
>             Project: Flink
>          Issue Type: Improvement
>          Components: Batch Connectors and Input/Output Formats
>            Reporter: Lars Bachmann
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> Currently it is not possible to set a compression codec for the AvroOutputFormat. 
> This improvement will provide a setter for the avro CodecFactory which is used by the
DataFileWriter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message