tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Siddharth Seth <ss...@apache.org>
Subject Re: Question about how TezConfiguration
Date Thu, 20 Mar 2014 00:50:23 GMT
Right, the configuration provided by the user gets written out and is made
available to the AM and tasks.

I¹m not sure what you mean by ŒI always get ³null²¹ - assuming this is
while trying to read a specific key. The Tez client filters out all
non-tez configuration parameters before writing out this Configuration -
which would cause an empty lookup.

Could you describe why you need to pass in a Configuration - typically,
I/P/O should have their own payload - which is what they use to configure
themselves.


On 3/19/14, 3:57 PM, "Chen He" <airbots@gmail.com> wrote:

>Dear Tez developers:
>
>I met a problem when I was working on TezLocalRunner. I need to know how
>user's configuration is passed to the YarnTezDagChild so that it can load
>it using:
>TezUtils.addUserSpecifiedTezConfiguration(conf);
>
>Here is my understanding:
>
>1) Users create their TezConfiguration and the TezClientUtils will write
>it
>to staging directory;
>2) DAGAppMaster's main method load TezConfiguration using
>TezUtils.addUserSpecifiedTezConfiguration(conf);
>3) when the YarnTezDagChild starts to run, its main method also call
>TezUtils.addUserSpecifiedTezConfiguration(conf);
>
>I met a problem that I always get "null" whenever I call the
>TezUtils.addUserSpecifiedTezConfiguration(conf) in the TezClient.
>
>Any reply will be appreciated!
>
>Regards!
>
>Chen



Mime
View raw message