spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zongheng Yang <zonghen...@gmail.com>
Subject Re: preferred Hive/Hadoop environment for generating golden test outputs
Date Fri, 18 Jul 2014 03:04:14 GMT
Hi Will,

These three environment variables are needed [1].

I have had success with Hive 0.12 and Hadoop 1.0.4. For Hive, getting
the source distribution seems to be required. Docs contribution will
be much appreciated!

[1] https://github.com/apache/spark/tree/master/sql#other-dependencies-for-developers

Zongheng

On Thu, Jul 17, 2014 at 7:51 PM, Will Benton <willb@redhat.com> wrote:
> Hi all,
>
> What's the preferred environment for generating golden test outputs for new Hive tests?
 In particular:
>
> * what Hadoop version and Hive version should I be using,
> * are there particular distributions people have run successfully, and
> * are there any system properties or environment variables (beyond HADOOP_HOME, HIVE_HOME,
and HIVE_DEV_HOME) I need to set before running the suite?
>
> I ask because I'm getting some errors while trying to add new tests and would like to
eliminate any possible problems caused by differences between what my environment offers and
what Spark expects.  (I'm currently running with the Fedora packages for Hadoop 2.2.0 and
a locally-built Hive 0.12.0.)  Since I'll only be using this for generating test outputs,
something as simple to set up as possible would be great.
>
> (Once I get something working, I'll be happy to write it up and contribute it as developer
docs.)
>
>
> thanks,
> wb

Mime
View raw message