crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rafal Wojdyla (JIRA)" <>
Subject [jira] [Created] (CRUNCH-470) Add hdfs/yarn minicluster crunch pipeline
Date Thu, 11 Sep 2014 10:27:33 GMT
Rafal Wojdyla created CRUNCH-470:

             Summary: Add hdfs/yarn minicluster crunch pipeline
                 Key: CRUNCH-470
             Project: Crunch
          Issue Type: Improvement
          Components: Core
    Affects Versions: 0.8.3
            Reporter: Rafal Wojdyla
            Assignee: Josh Wills
            Priority: Minor

Crunch currently has two pipelines:
* MemPipeline
* MRPipeline

MemPipeline is in-memory pipelines based on local in-memory mapreduce mode.
MRPipeline is distributed pipeline based on distributed MapReduce.

Using HDFS/YARN Minicluster it's possible to better emulate Hadoop cluster, and it could be
a 'final test' before running on the cluster.

This message was sent by Atlassian JIRA

View raw message