falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srikanth Sundarrajan <srik...@hotmail.com>
Subject RE: Forced Execution of Late Input Feed Instances
Date Wed, 01 Apr 2015 15:54:41 GMT
This is a case where one or more inputs is optional. This can be managed by configured the
input as optional. The job in this case would run every 6 hours with whatever input is available
without actually waiting for any inputs. We are currently discussing about more gating primitives.
Please see if the existing capability suffices. See "Optional Inputs" sections in http://falcon.apache.org/EntitySpecification.html
for additional details.

@dev is the right mailing list. We dont have the @user list created. This is to allow all
the dev to help with user queries as well.
Srikanth Sundarrajan

> From: eczech52@gmail.com
> Date: Wed, 1 Apr 2015 10:28:10 -0400
> Subject: Forced Execution of Late Input Feed Instances
> To: dev@falcon.apache.org
> Apologies if this isn't the right place for a question like this, but I
> thought it was worth asking before diving into the source code any more
> (and I'm not sure where else to go).
> Is there a way to force a process to run if one or more of its required
> input feed instances aren't yet available? By this I don't mean that the
> input feed instances arrive late, I mean that they're not present and they
> never will be (e.g. for sparse events recorded on a fairly small time scale
> like seconds or minutes).
> I'd like to tie a process that runs every 6 hours to an input feed with an
> hourly frequency but I'm not sure how to gracefully manage cases where one
> of the input hours is missing in an expected way, without causing the other
> 5 hours worth of data to be ignored.
> Is there a good place in the source code to look for where that behavior
> would be managed (to answer the question myself perhaps)?
> Thanks and sorry again for posing a user question to a dev list!
View raw message