phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-1199) Determine options for Phoenix 4.1.x supporting CDH 5.1
Date Sat, 23 Aug 2014 19:50:12 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-1199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14108115#comment-14108115
] 

James Taylor commented on PHOENIX-1199:
---------------------------------------

The Phoenix binaries will work against 0.98.1 and above, but *compiles" only on 0.98.4 or
above. This is an important distinction to make, as the majority of our users use our binaries
directly.

The fix that [~jesse_yates] made to use the new RPC changes will simply not be used at runtime
if a pre 0.98.4 HBase version is used. FWIW, this is a very important fix to prevent deadlocks
under load during mutable secondary index maintenance (PHOENIX-938). We made a conscious decision
to introduce a *compile time* dependency on 0.98.4, but ensured that at runtime 0.98.1 still
works.

The ServerName compilation issue will also have no impact at runtime - the sole occurrence
of constructing a ServerName is in ConnectionlessQueryServicesImpl which is not used anywhere
outside of unit tests. I've attached a small change that just tweaks the way in which it's
constructed which hopefully will get rid of that particular CDH compilation issue. If CDH
has the ServerName.parseServerName(String) method, then all should be well - would someone
mind verifying that?

I believe the issue that [~russell.jurney] ran into stems from a runtime problem he ran into
trying to use the phoenix-pig module with CDH 5.1. Correct me if I'm wrong, but if the Phoenix
jars just *worked*, you wouldn't have done down the path of trying to compile against CDH.
Is that correct, Russell?

The issue, I think, is that the phoenix-pig module was only compiled against the hadoop1 profile.
Instead, it should be compiled against both, with separate jars built and placed in the phoenix-4.1.0-bin/hadoop1
and phoenix-4.1.0-bin/hadoop2 directories (instead of phoenix-4.1.0-bin/common). In that case,
you'd simply use the hadoop2/phoenix-pig jar with CDH 5.1. This is likely the case with the
phoenix-flume module as well. If someone wants to volunteer to confirm that, it'd be much
appreciated.

Both the phoenix-pig module and the phoenix-flume module are in need of an owner and some
TLC (we're not currently using these modules at Salesforce.com, though, of course that may
always change). [~maghamravikiran] has expressed interested in maintaining these. If there's
not community interest in maintaining these modules, I think we should remove them for the
next release (this reminds me of the HBase contrib stuff I've heard about in the past).

I think compilation against other vendors distributions is a non goal. As Andrew pointed out
before, we have no control over what vendors include/don't include in their distribution.
I do think we should make a best effort attempt to have Phoenix *run* against other distributions.
However, ultimately this is controlled by the vendor and the best path to ensure this is to
get Phoenix into those distributions as Hortonworks has done. If you agree, let *your* vendor
know.

> Determine options for Phoenix 4.1.x supporting CDH 5.1 
> -------------------------------------------------------
>
>                 Key: PHOENIX-1199
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1199
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.1
>            Reporter: James Taylor
>         Attachments: PHOENIX-1199.patch
>
>
> Let's figure out the most painless way of supporting CDH 5.1 for Phoenix 4.1. I'm not
as concerned with compile-time, as we know we have a dependency on HBase 0.98.4 (to fix a
deadlock issue). However, this is not a runtime dependency. But the lack of the ServerName
is going to be a problem at runtime. Are there other problematic class references?
> What are our options? Should we try to get something in the next HBase release that'll
help (making constructors public, for example)? Or can we not use ServerName in the Phoenix
code? Are the old HBase APIs available still? You all would know better than me.
> Or should we just wait for the next patch release from Cloudera and ask nicely that they
make it more compatible? smile :-)
> [~apurtell], [~stack], [~lhofhansl], [~jesse_yates]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message