crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <>
Subject Re: getting crunch running on hadoop 2.0.0
Date Wed, 27 Jun 2012 04:31:01 GMT
Oh, and I wanted to mention: getting all of the unit tests to pass
under hadoop 2.0.0 was one of the blockers for getting Crunch added to

Now it's out of the way.

On Tue, Jun 26, 2012 at 9:26 PM, Josh Wills <> wrote:
> Hey all,
> That last commit from me "fixes" (in some loose sense of the word) the
> WordCountHBaseTest, which was the last test that we needed to get
> working in order to have all of the Crunch unit tests passing under
> Hadoop-2.0.0.
> The complexity stems from the fact that the WordCountHBaseTest starts
> up a mini hbase/mapreduce cluster in order to run, and I had to do
> some work to make sure that all of the proper dependencies and class
> definitions that were needed for the test case to pass were available
> to the mini YARN instance that runs w/HBaseTestingUtility under hadoop
> 2.0.0.
> Vinod, if you get a chance to take a look at the test case and see if
> there is anything I could have done to make it cleaner, I would be
> much obliged.
> J

Director of Data Science
Twitter: @josh_wills

View raw message