crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <>
Subject [jira] [Created] (CRUNCH-471) Add synchronization checks to DoFnIterator
Date Wed, 24 Sep 2014 21:11:34 GMT
Josh Wills created CRUNCH-471:

             Summary: Add synchronization checks to DoFnIterator
                 Key: CRUNCH-471
             Project: Crunch
          Issue Type: Bug
          Components: Core
    Affects Versions: 0.11.0
            Reporter: Josh Wills
            Assignee: Josh Wills
             Fix For: 0.12.0

I hit this one when I was working on a port of the Hadoop Streaming logic to Crunch: if a
DoFn emits records via some sort of asynchronous process (e.g., a daemon thread), it can cause
a problem for DoFnIterator, which doesn't have thread-safe controls over its underlying records
list. Easy fix is to make the accessor methods for CacheEmitter inside of DoFnIterator synchronized.

This message was sent by Atlassian JIRA

View raw message