falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srikanth Sundarrajan <srik...@hotmail.com>
Subject RE: Forced Execution of Late Input Feed Instances
Date Thu, 02 Apr 2015 09:34:11 GMT
This is something that we intend to build, but we dont have it yet in the system.

Regards
Srikanth Sundarrajan

----------------------------------------
> From: eczech52@gmail.com
> Date: Wed, 1 Apr 2015 12:23:19 -0400
> Subject: Re: Forced Execution of Late Input Feed Instances
> To: dev@falcon.apache.org
>
> Thanks Srikanth,
>
> That helps for sure, though I'll probably have to delay things
> unnecessarily to do that.
>
> Would a future gating primitive in the works cover a case like this?
>
> - We want a process instance to run right away if all data is present
> - We want the process to wait up to X minutes for late data before running
> anyways
>
> I hope I'm not missing anything but it seems like with optional inputs that
> I'll just have to make "right away" be some time further in the future
> (i.e. the time I'd like the process to run plus the X minute delay).
>
> Think there's any hope to pull that off with existing functionality?
>
> Thanks again!
>
>
>
> On Wed, Apr 1, 2015 at 11:54 AM, Srikanth Sundarrajan <sriksun@hotmail.com>
> wrote:
>
>> This is a case where one or more inputs is optional. This can be managed
>> by configured the input as optional. The job in this case would run every 6
>> hours with whatever input is available without actually waiting for any
>> inputs. We are currently discussing about more gating primitives. Please
>> see if the existing capability suffices. See "Optional Inputs" sections in
>> http://falcon.apache.org/EntitySpecification.html for additional details.
>>
>> @dev is the right mailing list. We dont have the @user list created. This
>> is to allow all the dev to help with user queries as well.
>> Regards
>> Srikanth Sundarrajan
>>
>> ----------------------------------------
>>> From: eczech52@gmail.com
>>> Date: Wed, 1 Apr 2015 10:28:10 -0400
>>> Subject: Forced Execution of Late Input Feed Instances
>>> To: dev@falcon.apache.org
>>>
>>> Apologies if this isn't the right place for a question like this, but I
>>> thought it was worth asking before diving into the source code any more
>>> (and I'm not sure where else to go).
>>>
>>> Is there a way to force a process to run if one or more of its required
>>> input feed instances aren't yet available? By this I don't mean that the
>>> input feed instances arrive late, I mean that they're not present and
>> they
>>> never will be (e.g. for sparse events recorded on a fairly small time
>> scale
>>> like seconds or minutes).
>>>
>>> I'd like to tie a process that runs every 6 hours to an input feed with
>> an
>>> hourly frequency but I'm not sure how to gracefully manage cases where
>> one
>>> of the input hours is missing in an expected way, without causing the
>> other
>>> 5 hours worth of data to be ignored.
>>>
>>> Is there a good place in the source code to look for where that behavior
>>> would be managed (to answer the question myself perhaps)?
>>>
>>> Thanks and sorry again for posing a user question to a dev list!
>>
>>
 		 	   		  
Mime
View raw message