giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roman Shaposhnik <ro...@shaposhnik.org>
Subject Re: Release 1.1.0
Date Wed, 17 Sep 2014 17:57:55 GMT
Hi!

On Mon, Sep 15, 2014 at 10:26 AM, Claudio Martella
<claudio.martella@gmail.com> wrote:
> Hi all,
>
> I've talked to Roman about the reasons why 1.1.0 fell in the backlog. I
> think it was quite a shame we managed the last voting round.
>
> Roman should be capable better than me to summarize and explain the status,
> but for what I understand we should be able to release with builds for
> hadoop 1 without problems, and only core and examples for hadoop 2.

Let me try to recap what is known so far.



Here's what I'm getting when running on a freshly
booted Amazon EC2 VM of Ubuntu 14.04:

$ umask 0022
$ curl http://people.apache.org/~rvs/giraph-1.1.0-RC0/giraph-dist-1.1.0-src.tar.bz2
| tar xjvf -
$ cd giraph-1.1.0
$ rm -rf ~/.m2
$ mvn --version
Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4;
2014-08-11T20:58:10+00:00)
Maven home: /home/ubuntu/apache-maven-3.2.3
Java version: 1.7.0_65, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-7-openjdk-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.13.0-35-generic", arch: "amd64", family: "unix"
$ mvn clean install -Phadoop_1
SUCCESS

$ rm -rf ~/.m2
$ mvn clean install -Phadoop_2
[INFO] Apache Giraph Parent ............................... SUCCESS [ 19.361 s]
[INFO] Apache Giraph Core ................................. SUCCESS [09:37 min]
[INFO] Apache Giraph .............. SUCCESS [10:14 min]

For everything else tests FAIL (expected)

$ mvn clean install -Phadoop_2 -DskipTests
SUCCESS

What this means is that I can't reproduce build failures
in hadoop_1 profile. There are *expected* test failures
in hadoop_2 profile on test and I suggest we RELNOTE
those.

All the other profiles may fail or not -- that wouldn't make
a difference. Or at least it wasn't supposed to (I thought
we agreed that hadoop_[12] were the only official
ones).

To summarize, hadoop 1 works all, hadoop 2 build works,
but tests fail ONLY for some of the I/O component (core
and example tests pass just fine with hadoop 2 as well).

Another point is that things like HBase, Hive/HCatalog when
compiled against Hadoop2 manually work just fine even in tests.

Why do I think we shouldn't just disable tests in hadoop_2
is the fact that they can be useful IFF you're willing
to build dependencies for them manually. IOW,  if you go a
trouble of actually building artifacts of HBase and Hive/HCatalog
against Hadoop 2 and have *those* artifacts in your local
.m2 -- than even tests pass.

This is exactly what Bigtop does and that's how I know
this stuff works with Hadoop 2 just fine. It just requires
a really complicated build process at this point.

> Roman explained that the problems with some inputformat like HBase etc. are
> due to hadoop 2, and not necessarily specific to giraph. Maybe Roman can
> explain better.
>
> So, I'm proposing to vote for 1.1.0 again. What do you think?

I think restarting the vote would be the right thing to do. Worst case
scenario -- we will uncover real issues and will spin up a new RC.

Thanks,
Roman.

Mime
View raw message