spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Nunez <>
Subject Re: Issues with HDP
Date Mon, 04 Aug 2014 17:39:13 GMT
Hmm. Fair enough. I hadn¹t given that answer much thought and on
reflection think you¹re right in that a profile would just be a bad hack.

On 8/4/14, 10:35, "Sean Owen" <> wrote:

>What would such a profile do though? In general building for a
>specific vendor version means setting hadoop.verison and/or
>yarn.version. Any hard-coded value is unlikely to match what a
>particular user needs. Setting protobuf versions and so on is already
>done by the generic profiles.
>In a similar vein, I am not clear on why there's a mapr profile in the
>build. Its versions are about to be out of date and won't work with
>upcoming Hbase changes for example.
>(Elsewhere in the build I think it wouldn't hurt to clear out
>cloudera-specific profiles and releases too -- they're not in the pom
>but are in the distribution script. It's the vendor's problem.)
>This isn't any argument about being purist but just that I am not sure
>these are things that the project can meaningfully bother with.
>It makes sense to set vendor repos in the pom for convenience, and
>makes sense to run smoke tests in Jenkins against particular versions.
>On Mon, Aug 4, 2014 at 6:21 PM, Steve Nunez <>
>> I don¹t think there is an hwx profile, but there probably should be.

NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message