hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brahma Reddy Battula (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-10606) NodeManager cannot launch container when using RawLocalFileSystem for fs.file.impl
Date Tue, 12 Apr 2016 06:55:25 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-10606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Brahma Reddy Battula updated HADOOP-10606:
------------------------------------------
    Assignee:     (was: Brahma Reddy Battula)

> NodeManager cannot launch container when using RawLocalFileSystem for fs.file.impl
> ----------------------------------------------------------------------------------
>
>                 Key: HADOOP-10606
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10606
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs, io, util
>    Affects Versions: 2.4.0
>         Environment: The environment does not matter with this issue. But I use Windows
8 64bits, Open JDK 7.0.
>            Reporter: BoYang
>            Priority: Critical
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> Node manager failed to launch container when I set fs.file.impl to org.apache.hadoop.fs.RawLocalFileSystem
in core-site.xml.
> The log is:
> WARN ContainersLauncher #11 org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch
- Failed to launch container.
> java.lang.ClassCastException: org.apache.hadoop.fs.RawLocalFileSystem cannot be cast
to org.apache.hadoop.fs.LocalFileSystem
> 	at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:339)
> 	at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:270)
> 	at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:344)
> 	at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150)
> 	at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLogPathForWrite(LocalDirsHandlerService.java:307)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:185)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> 	at java.lang.Thread.run(Thread.java:722)
> The issue is in hadoop-common-project\hadoop-common\src\main\java\org\apache\hadoop\fs\LocalDirAllocator.java.
It invokes FileSystem.getLocal(), which tries to convert the FileSystem to LocalFileSystem
and will fail when FileSystem object is RawLocalFileSystem (RawLocalFileSystem is not a sub
class of LocalFileSystem).
>   public static LocalFileSystem getLocal(Configuration conf)
>     throws IOException {
>     return (LocalFileSystem)get(LocalFileSystem.NAME, conf);
>   }
> The fix for LocalDirAllocator.java seems to be invoking LocalFileSystem.get() instead
of LocalFileSystem.getLocal()?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message