beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marian Dvorsky (JIRA)" <>
Subject [jira] [Created] (BEAM-3874) Switch AvroIO sink default codec to Snappy
Date Sun, 18 Mar 2018 15:43:00 GMT
Marian Dvorsky created BEAM-3874:

             Summary: Switch AvroIO sink default codec to Snappy
                 Key: BEAM-3874
             Project: Beam
          Issue Type: Improvement
          Components: io-java-avro
            Reporter: Marian Dvorsky
            Assignee: Eugene Kirpichov

AvroIO currently uses [CodecFactory|].[deflateCodec|](6)
as the default codec for writes.

That compresses well, but is quite expensive.

Snappy codec offers sparser, but much faster compression, and is typically a better CPU/storage
tradeoff except for very long lived files. 

We should consider switching the default to Snappy.

This message was sent by Atlassian JIRA

View raw message