phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Istvan Toth <st...@apache.org>
Subject [DISCUSS] HBase 2.2.5 public maven artifacts are incompatible with Hadoop 3
Date Tue, 07 Jul 2020 08:39:23 GMT
Hi!

(https://issues.apache.org/jira/browse/PHOENIX-5993 is open for this, but I
am repeating it here for greater visibility)

The HBase artifacts downloadable from maven central are built with Hadoop
2.x

However, up to now, we could use these artifacts with Hadoop 3.0 and they
worked. (Or we just didn't hit the incompatibilities in our test suite.)

This seems to have changed with 2.2.5, as the public maven artifact doesn't
work with Hadoop 3.0.3 or 3.1.2 .

This is a known issue in HBase, and documented, but this means that

Any client JAR we'd build with Hbase 2.2.5 would have the same problem.
We cannot run our tests with Hbase 2.2.5
HBase's suggested solution is to rebuild HBase with the Hadoop version used
in the cluster, and use that. This, however, doesn't fit into our test or
distribution process.

I'm looking for your input on how to fix this.

The options I could think of are:

- Ask HBase to fix it for us (it doesn't seem to be a priority)
 - HBase could publish artifacts for Hadoop3
 - Or could try to stick the common subset of Hadoop APIs
- Re-publish the official HBase releases built with Hadoop 3 under the
org.apache.phoenix groupId
- Hack up our build system to rebuild HBase from source, and use that for
Phoenix

None of these are particularly compelling, but I expect that this will have
to be solved somehow if we want to keep up with future Hadoop releases.

Looking forward to hearing your ideas and opinion on this

Istvan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message