crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <>
Subject [jira] [Resolved] (CRUNCH-268) Crunch's internal Avro tuple schemas should have stable names
Date Sat, 21 Sep 2013 20:38:51 GMT


Josh Wills resolved CRUNCH-268.

       Resolution: Fixed
    Fix Version/s: 0.8.0

Committed to master.
> Crunch's internal Avro tuple schemas should have stable names
> -------------------------------------------------------------
>                 Key: CRUNCH-268
>                 URL:
>             Project: Crunch
>          Issue Type: Bug
>          Components: Core, IO
>    Affects Versions: 0.7.0
>            Reporter: Josh Wills
>            Assignee: Josh Wills
>             Fix For: 0.8.0
>         Attachments: CRUNCH-268.patch, CRUNCH-268v2.patch
> A long time ago, I made a change that used random names for the custom Avro schemas that
Crunch generates for processing tuple types (pairs, trips, etc.). I recently hit a use case
where that randomization burned me when I was re-running some pipelines over checkpointed
data that I serialized using Crunch's Avro schemas (Pair, in particular), so I think that
we should change the tuple schemas to have stable names based on their constituent field schemas
via an MD5 hash.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message