tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hitesh Shah <hit...@apache.org>
Subject Re: Tez Jars/Lib Exception
Date Tue, 23 Jul 2013 20:22:42 GMT
Hi Achal, 

I am not sure if that use-case is doable at the moment. It seems like what you are asking
for is the ability to create a jar on the fly if a jar is not found. Feel free to go ahead
and open a jira to address this use-case.

In any case, the code does not handle a jar not being found. I filed TEZ-315 if you want to
take that up. 

thanks
-- Hitesh


On Jul 23, 2013, at 12:13 PM, Achal Soni wrote:

> I was referring to this:
> 
>    String jarPath = ClassUtil.findContainingJar(getClass());
> 
>    Path remoteJarPath = remoteFs.makeQualified(
> 
>        new Path(remoteStagingDir, "dag_job.jar"));
> 
>    remoteFs.copyFromLocalFile(new Path(jarPath), remoteJarPath);
> 
>    FileStatus jarFileStatus = remoteFs.getFileStatus(remoteJarPath);
> 
> 
>    Map<String, LocalResource> commonLocalResources =
> 
>        new HashMap<String, LocalResource>();
> 
>    LocalResource dagJarLocalRsrc = LocalResource.newInstance(
> 
>        ConverterUtils.getYarnUrlFromPath(remoteJarPath),
> 
>        LocalResourceType.FILE,
> 
>        LocalResourceVisibility.APPLICATION,
> 
>        jarFileStatus.getLen(),
> 
>        jarFileStatus.getModificationTime());
> 
>    commonLocalResources.put("dag_job.jar", dagJarLocalRsrc);
> 
> 
> This causes issues in Eclipse because there is no jar per se.
> 
> 
> On Mon, Jul 22, 2013 at 11:25 PM, Hitesh Shah <hitesh@apache.org> wrote:
> 
>> I am not sure what you mean by the jar containing the submission. All the
>> tez jars ( from the distribution tarball ) are added as local resources for
>> the AM as well as all vertices by default.
>> 
>> -- Hitesh
>> 
>> On Jul 22, 2013, at 10:59 PM, Achal Soni wrote:
>> 
>>> Ah I see. And is it necessary to add the jar containing the submission
>> as a
>>> local resource? This is causing issues when running from Eclipse. I guess
>>> it is necessary for the map and reduce classes?
>>> 
>>> - Achal
>> 
>> 


Mime
View raw message