samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Pan \(Data Infrastructure\)" <yi...@linkedin.com>
Subject Re: Review Request 43732: Implemented AvroDataFileHdfsWriter
Date Wed, 02 Mar 2016 06:28:21 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43732/#review121594
-----------------------------------------------------------



Overall lgtm. There is some problem in the last file in the diff uploaded. Could you try to
rebase and upload again?Thanks!


gradle/dependency-versions.gradle (line 23)
<https://reviews.apache.org/r/43732/#comment183340>

    Please rebase w/ the latest master branch.


- Yi Pan (Data Infrastructure)


On Feb. 25, 2016, 7:39 p.m., Edi Bice wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43732/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2016, 7:39 p.m.)
> 
> 
> Review request for samza.
> 
> 
> Repository: samza
> 
> 
> Description
> -------
> 
> https://issues.apache.org/jira/browse/SAMZA-876 
> 
> Implemented AvroDataFileHdfsWriter fashioned loosely after BinarySequenceFileHDFSWriter.
> 
> 
> Diffs
> -----
> 
>   docs/learn/documentation/versioned/hdfs/producer.md cfd22c6 
>   docs/learn/documentation/versioned/jobs/configuration-table.html 6705530 
>   gradle/dependency-versions.gradle 52e25aa 
>   samza-hdfs/src/main/scala/org/apache/samza/system/hdfs/HdfsConfig.scala 7993119 
>   samza-hdfs/src/main/scala/org/apache/samza/system/hdfs/writer/AvroDataFileHdfsWriter.scala
PRE-CREATION 
>   samza-hdfs/src/test/resources/samza-hdfs-test-batch-job-avro.properties PRE-CREATION

>   samza-hdfs/src/test/resources/samza-hdfs-test-job-avro.properties PRE-CREATION 
>   samza-hdfs/src/test/scala/org/apache/samza/system/hdfs/TestHdfsSystemProducerTestSuite.scala
c4b04a1 
> 
> Diff: https://reviews.apache.org/r/43732/diff/
> 
> 
> Testing
> -------
> 
> Two JUnit tests similar to the Text/BinarySequenceFileHdfsWriter ones. In addition I've
been using AvroDataFileHdfsWriter at the end of my pipeline. I feed the generated avro files
to Apache Samoa. Have processed millions of records successfully.
> 
> 
> Thanks,
> 
> Edi Bice
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message