falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Czech <eczec...@gmail.com>
Subject Re: Forced Execution of Late Input Feed Instances
Date Thu, 02 Apr 2015 14:01:11 GMT
Ah ok, good to know!  Not to hard to work around that for now, and I
appreciate the insight.

On Thu, Apr 2, 2015 at 5:34 AM, Srikanth Sundarrajan <sriksun@hotmail.com>
wrote:

> This is something that we intend to build, but we dont have it yet in the
> system.
>
> Regards
> Srikanth Sundarrajan
>
> ----------------------------------------
> > From: eczech52@gmail.com
> > Date: Wed, 1 Apr 2015 12:23:19 -0400
> > Subject: Re: Forced Execution of Late Input Feed Instances
> > To: dev@falcon.apache.org
> >
> > Thanks Srikanth,
> >
> > That helps for sure, though I'll probably have to delay things
> > unnecessarily to do that.
> >
> > Would a future gating primitive in the works cover a case like this?
> >
> > - We want a process instance to run right away if all data is present
> > - We want the process to wait up to X minutes for late data before
> running
> > anyways
> >
> > I hope I'm not missing anything but it seems like with optional inputs
> that
> > I'll just have to make "right away" be some time further in the future
> > (i.e. the time I'd like the process to run plus the X minute delay).
> >
> > Think there's any hope to pull that off with existing functionality?
> >
> > Thanks again!
> >
> >
> >
> > On Wed, Apr 1, 2015 at 11:54 AM, Srikanth Sundarrajan <
> sriksun@hotmail.com>
> > wrote:
> >
> >> This is a case where one or more inputs is optional. This can be managed
> >> by configured the input as optional. The job in this case would run
> every 6
> >> hours with whatever input is available without actually waiting for any
> >> inputs. We are currently discussing about more gating primitives. Please
> >> see if the existing capability suffices. See "Optional Inputs" sections
> in
> >> http://falcon.apache.org/EntitySpecification.html for additional
> details.
> >>
> >> @dev is the right mailing list. We dont have the @user list created.
> This
> >> is to allow all the dev to help with user queries as well.
> >> Regards
> >> Srikanth Sundarrajan
> >>
> >> ----------------------------------------
> >>> From: eczech52@gmail.com
> >>> Date: Wed, 1 Apr 2015 10:28:10 -0400
> >>> Subject: Forced Execution of Late Input Feed Instances
> >>> To: dev@falcon.apache.org
> >>>
> >>> Apologies if this isn't the right place for a question like this, but I
> >>> thought it was worth asking before diving into the source code any more
> >>> (and I'm not sure where else to go).
> >>>
> >>> Is there a way to force a process to run if one or more of its required
> >>> input feed instances aren't yet available? By this I don't mean that
> the
> >>> input feed instances arrive late, I mean that they're not present and
> >> they
> >>> never will be (e.g. for sparse events recorded on a fairly small time
> >> scale
> >>> like seconds or minutes).
> >>>
> >>> I'd like to tie a process that runs every 6 hours to an input feed with
> >> an
> >>> hourly frequency but I'm not sure how to gracefully manage cases where
> >> one
> >>> of the input hours is missing in an expected way, without causing the
> >> other
> >>> 5 hours worth of data to be ignored.
> >>>
> >>> Is there a good place in the source code to look for where that
> behavior
> >>> would be managed (to answer the question myself perhaps)?
> >>>
> >>> Thanks and sorry again for posing a user question to a dev list!
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message