tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hitesh Shah <hit...@apache.org>
Subject Re: Tez configuration initialization ignoring JobConfigurable
Date Fri, 06 Jun 2014 16:57:35 GMT
Hi Subroto

Could you provide some more context on what you are trying to do? Are you
trying to run MR-on-Tez? or a native Tez job?
If you could provide us with some code showing what you are trying to do,
we can help further. There are probably some bugs in the MR compatibility
that we may have not come across.

— Hitesh

On Fri, Jun 6, 2014 at 6:53 AM, Subroto Sanyal <sanyalsubroto@gmail.com>

> Hi,
> Tez has utility which created Configuration object from the payload:
> TezUtils.createConfFromUserPayload(byte[] payload); this method returns a
> Configuration object even though the serialized byte[] can be of type
> JobConf.
> Once we get the Configuration we try to  create few object using
> ReflectionUtil.newInstance(class, conf). ReflectionUtil.newInstance makes a
> check whether the conf is instance of "org.apache.hadoop.mapred.JobConf"
> and accordingly invokes the "configure" method.
> This behavior is not working  anymore in Tez scenario. One simple scenario
> when user defines a custom "RawComparator" and makes it "JobConfigurable"
> but, org.apache.tez.runtime.library.common.sort.impl.ExternalSorter doesn't
> care if the configuration could be instance of "org.apache.hadoop.mapred.
> JobConf"
> Please let me know if there is a problem with Tez or there exist lack of my
> understanding about how objects should be created in Tez  :-)
> --
> Cheers,
> *Subroto Sanyal*

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message