falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Seetharam Venkatesh <venkat...@innerzeal.com>
Subject Re: support webhdfs for feed replication
Date Wed, 09 Apr 2014 07:32:19 GMT
I think Oozie needs to be configured with the schemes that you are
interested in.

oozie.service.HadoopAccessorService.supported.filesystems must have
webhdfs. I'll also check in a day to test this scenario.


On Tue, Apr 1, 2014 at 11:00 PM, Praveen <sadhu.apps@gmail.com> wrote:

> Thanks for the response.  Scheduling directly in oozie seems to work fine.
>  Also, using hdfs for target works fine as well.   Am thinking if the error
> stack is misleading.
>
>
> On Tue, Apr 1, 2014 at 10:56 PM, Samarth Gupta <samarthgupta007@gmail.com
> >wrote:
>
> > @praveen , can u try directly scheduling in oozie ?  other ppl have also
> > faced similar issue
> >
> >
> http://grokbase.com/t/cloudera/cdh-user/1334qxqcyk/error-when-running-oozie-mapreduce-example
> >
> > as Srikanth mentioned ,  webhdfs is supported by falcon.
> >
> > Thanks:
> > Samarth
> >
> >
> > On Wed, Apr 2, 2014 at 11:22 AM, Srikanth Sundarrajan
> > <sriksun@hotmail.com>wrote:
> >
> > > Seems like an issue with connection to JT from the oozie server. As
> such
> > > webhdfs is supported via falcon for replication.
> > > RegardsSrikanth Sundarrajan
> > >
> > > > Date: Tue, 1 Apr 2014 13:28:57 -0700
> > > > Subject: support webhdfs for feed replication
> > > > From: sadhu.apps@gmail.com
> > > > To: dev@falcon.incubator.apache.org
> > > >
> > > > Hi,
> > > >
> > > >
> > > > I am new this group and wanted to check with the dev if webhdfs is
> > > > something the current trunk supports for feed replication.   I have
> > tried
> > > > to setup a cluster with webhdfs, however, oozie seems to fail to
> > > initialize
> > > > cluster with the following exception.
> > > >
> > > >
> > > > 2014-04-01 06:20:02,755  WARN ActionStartXCommand:542 -
> USER[bigdatad]
> > > > GROUP[-] TOKEN[] APP[FALCON_FEED_REPLICATION_distcp-feed]
> > > > JOB[0000149-140331222820297-oozie-bigd-W] AC
> > > >
> > > > TION[0000149-140331222820297-oozie-bigd-W@recordsize] Error starting
> > > action
> > > > [recordsize]. ErrorType [TRANSIENT], ErrorCode [JA009], Message
> [JA009:
> > > > Cannot initialize Cluster
> > > >
> > > > . Please check your configuration for mapreduce.framework.name and
> the
> > > > correspond server addresses.]
> > > >
> > > > org.apache.oozie.action.ActionExecutorException: JA009: Cannot
> > initialize
> > > > Cluster. Please check your configuration for
> > mapreduce.framework.nameand
> > > > the correspond server add
> > > >
> > > > resses.
> > > >
> > > >         at
> > > >
> > >
> >
> org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:412)
> > > >
> > > >         at
> > > >
> > >
> >
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392)
> > > >
> > > >         at
> > > >
> > >
> >
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:794)
> > > >
> > > >         at
> > > >
> > >
> >
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:945)
> > > >
> > > >         at
> > > >
> > >
> >
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:215)
> > > >
> > > >         at
> > > >
> > >
> >
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:60)
> > > >
> > > >         at org.apache.oozie.command.XCommand.call(XCommand.java:280)
> > > >
> > > >         at
> > > >
> > >
> >
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
> > > >
> > > >         at
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > > >
> > > >         at
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > > >
> > > >         at java.lang.Thread.run(Thread.java:744)
> > > >
> > > > Caused by: java.io.IOException: Cannot initialize Cluster. Please
> check
> > > > your configuration for mapreduce.framework.name and the correspond
> > > server
> > > > addresses.
> > > >
> > > >         at
> > > org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
> > > >
> > > >         at
> org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
> > > >
> > > >         at
> org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
> > > >
> > > >         at
> org.apache.hadoop.mapred.JobClient.init(JobClient.java:470)
> > > >
> > > >         at
> > org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:449)
> > > >
> > > >         at
> > > >
> > >
> >
> org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:371)
> > > >
> > > >         at
> > > >
> > >
> >
> org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:369)
> > > >
> > > >         at java.security.AccessController.doPrivileged(Native Method)
> > > >
> > > >         at javax.security.auth.Subject.doAs(Subject.java:415)
> > > >
> > > >         at
> > > >
> > >
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> > > >
> > > >         at
> > > >
> > >
> >
> org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:369)
> > > >
> > > >         at
> > > >
> > >
> >
> org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:988)
> > > >
> > > >         at
> > > >
> > >
> >
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:743)
> > > >
> > > >         ... 8 more
> > > >
> > > >
> > > > Thanks!
> > >
> > >
> >
>



-- 
Regards,
Venkatesh

"Perfection (in design) is achieved not when there is nothing more to add,
but rather when there is nothing more to take away."
- Antoine de Saint-Exupéry

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message