falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Clum <joshc...@gmail.com>
Subject Process NPE
Date Wed, 13 Aug 2014 16:54:43 GMT
Hi, I'm getting the following NPE when i launch with "falcon entity -type
process -schedule -name raw-cc-bp-ratio-lcms-comp-process". At the bottom
is my process which has no inputs.

If I look at
https://github.com/apache/incubator-falcon/blob/branch-0.5/common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
it seems that this should never happen because process.getInputs() ==
null should
be true and should avoid the NPE. Any thoughts?

2014-08-13 16:49:05,152 ERROR -
[6221571@qtp-1696932027-3:vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
a38b5528-cae0-4b07-a8fc-482661e467a9] ~ Unable to schedule workflow
(AbstractSchedulableEntityManager:60)
java.lang.NullPointerException
at
org.apache.falcon.entity.ProcessHelper.getStorageType(ProcessHelper.java:57)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.isTableStorageType(OozieProcessWorkflowBuilder.java:646)
 at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.shouldSetupHiveConfiguration(OozieProcessWorkflowBuilder.java:641)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createWorkflow(OozieProcessWorkflowBuilder.java:606)
 at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createDefaultCoordinator(OozieProcessWorkflowBuilder.java:291)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.getCoordinators(OozieProcessWorkflowBuilder.java:206)
 at
org.apache.falcon.workflow.OozieWorkflowBuilder.map(OozieWorkflowBuilder.java:142)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.newWorkflowSchedule(OozieProcessWorkflowBuilder.java:111)
 at
org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:119)
at
org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:68)
 at
org.apache.falcon.resource.AbstractSchedulableEntityManager.schedule(AbstractSchedulableEntityManager.java:57)
at
org.apache.falcon.resource.SchedulableEntityManager.schedule(SchedulableEntityManager.java:85)
 at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.falcon.resource.channel.IPCChannel.invoke(IPCChannel.java:48)
 at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$7.doExecute(SchedulableEntityManagerProxy.java:324)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$EntityProxy.execute(SchedulableEntityManagerProxy.java:410)
 at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule_aroundBody10(SchedulableEntityManagerProxy.java:326)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$AjcClosure11.run(SchedulableEntityManagerProxy.java:1)
 at
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
at
org.apache.falcon.aspect.AbstractFalconAspect.logAround(AbstractFalconAspect.java:50)
 at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule(SchedulableEntityManagerProxy.java:310)
at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
 at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
 at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
 at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
 at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
 at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
 at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
 at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
 at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
 at
org.apache.falcon.security.BasicAuthFilter$2.doFilter(BasicAuthFilter.java:183)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:392)
 at
org.apache.falcon.security.BasicAuthFilter.doFilter(BasicAuthFilter.java:221)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
 at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
 at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
 at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
 at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
2014-08-13 16:49:05,154 ERROR -
[6221571@qtp-1696932027-3:vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
a38b5528-cae0-4b07-a8fc-482661e467a9] ~ Failure reason
(FalconWebException:40)
java.lang.NullPointerException
at
org.apache.falcon.entity.ProcessHelper.getStorageType(ProcessHelper.java:57)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.isTableStorageType(OozieProcessWorkflowBuilder.java:646)
 at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.shouldSetupHiveConfiguration(OozieProcessWorkflowBuilder.java:641)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createWorkflow(OozieProcessWorkflowBuilder.java:606)
 at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.createDefaultCoordinator(OozieProcessWorkflowBuilder.java:291)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.getCoordinators(OozieProcessWorkflowBuilder.java:206)
 at
org.apache.falcon.workflow.OozieWorkflowBuilder.map(OozieWorkflowBuilder.java:142)
at
org.apache.falcon.workflow.OozieProcessWorkflowBuilder.newWorkflowSchedule(OozieProcessWorkflowBuilder.java:111)
 at
org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:119)
at
org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:68)
 at
org.apache.falcon.resource.AbstractSchedulableEntityManager.schedule(AbstractSchedulableEntityManager.java:57)
at
org.apache.falcon.resource.SchedulableEntityManager.schedule(SchedulableEntityManager.java:85)
 at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.falcon.resource.channel.IPCChannel.invoke(IPCChannel.java:48)
 at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$7.doExecute(SchedulableEntityManagerProxy.java:324)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$EntityProxy.execute(SchedulableEntityManagerProxy.java:410)
 at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule_aroundBody10(SchedulableEntityManagerProxy.java:326)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$AjcClosure11.run(SchedulableEntityManagerProxy.java:1)
 at
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
at
org.apache.falcon.aspect.AbstractFalconAspect.logAround(AbstractFalconAspect.java:50)
 at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule(SchedulableEntityManagerProxy.java:310)
at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
 at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
 at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
 at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
 at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
 at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
 at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
 at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
 at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
 at
org.apache.falcon.security.BasicAuthFilter$2.doFilter(BasicAuthFilter.java:183)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:392)
 at
org.apache.falcon.security.BasicAuthFilter.doFilter(BasicAuthFilter.java:221)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
 at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
 at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
 at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
 at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
2014-08-13 16:49:05,154 ERROR -
[6221571@qtp-1696932027-3:vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
a38b5528-cae0-4b07-a8fc-482661e467a9] ~ Action failed: Bad Request
Error:null
 (FalconWebException:66)
2014-08-13 16:49:05,155 INFO  -
[6221571@qtp-1696932027-3:vagrant:POST//entities/schedule/process/raw-cc-bp-ratio-lcms-comp-process
a38b5528-cae0-4b07-a8fc-482661e467a9] ~ {Action:schedule,
Dimensions:{entityType=process, colo=*,
entityName=raw-cc-bp-ratio-lcms-comp-process}, Status: FAILED,
Time-taken:161608565 ns} (METRIC:32)

My process looks like this:

<process name="raw-cc-bp-ratio-lcms-comp-process"
xmlns="uri:falcon:process:0.1">

  <tags>type=raw</tags>

  <clusters>
    <cluster name="minicluster">
      <validity start="2014-08-13T00:00Z" end="2016-01-01T00:00Z"/>
    </cluster>
  </clusters>

  <parallel>1</parallel>
  <order>FIFO</order>
  <timeout>hours(2)</timeout>
  <frequency>days(1)</frequency>

  <outputs>
    <output name="output" feed="raw-cc-bp-ratio-lcms-comp-feed"
instance="now(0,0)" />
  </outputs>

  <properties>
...
  </properties>

  <workflow name="sqoop_wf" version="2.0.0" engine="oozie"
path="/apps/huron/ingest/standard_ingest/sqoop_wf.xml" />

  <retry policy="periodic" delay="minutes(10)" attempts="3" />

</process>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message