tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Siddharth Seth <ss...@apache.org>
Subject Re: Helpers for MAPREDUCE-3727 scenario
Date Tue, 10 Mar 2015 19:32:51 GMT
Tez does provide some config helpers - to translate configs between MR and
Tez. I'm not sure which downstream projects actually make use of these. We
could add some cleanup code here / or a new helper - but it'll be up to the
users to actually make use of this in the correct place.
For the specific property (mapreduce.job.credentials.binary) - Tez uses
it's own property for this - which I believe Pig / Hive setup. If I
understand the usage correctly - Oozie should only be loading this property
in the launcher, which should have a separate configuration from the actual
job. Does Oozie leak these configs into the configs for the main job ?

On Tue, Mar 10, 2015 at 8:32 AM, Jonathan Eagles <jeagles@gmail.com> wrote:

> MapReduce (MAPREDUCE-3727) and Pig on Tez (PIG-4333) have implemented
> a work around to unset the mapreduce.job.credentials.binary property.
> Now hive has hit the same scenario this week (no jira yet that I see).
> While Tez dones generally handle mapreduce properties, should we
> provide a config cleaning helper that will help unify the config
> cleaning that will protect our users?
>
> Jon
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message