falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajay Yadav (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (FALCON-722) Add SLA for processes
Date Tue, 04 Nov 2014 12:03:33 GMT

    [ https://issues.apache.org/jira/browse/FALCON-722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196030#comment-14196030
] 

Ajay Yadav edited comment on FALCON-722 at 11/4/14 12:03 PM:
-------------------------------------------------------------

I thought more on this and here are my thoughts on it:

*Burden for user?*
All the values are optional and user can configure only a few, so it shouldn't be a burden
on the user to configure a process. User can choose to not configure any SLA if he wishes
to do so.

*{{slaStart}}*
It is required to alert a user when the process hasn't kickstarted. 

*Won't sla for feeds(input feeds) solve for it?*
* sla for feeds is optional and hence it's not guaranteed that it will trigger.
* Configuring sla for one process is more convenient than configuring sla for all the feeds.
* It provides a very convenient reporting number to tell how many times the process input
was delayed. I can get process instance start times but I won't have a number to draw acceptable
boundaries. Both reporting dashboard and alerting software will be easy to keep consistent
if we use slaStart and read it from here. 
* What if it's a process with monthly frequency and someone puts an input feed's retention
not long enough. The feed won't alert in that case and process won't start either. I actually
witnessed it, a monthly process started at 6th of every month and used to consume a daily
feed for entire last month. Someone updated the daily feed's retention to 30 days and the
process missed SLA.

*{{slaEnd}}*
* I think slaEnd is primary component to monitor whether the process finished on time expected.
* Some processes may not have output feeds at all and hence it is important to have it.
This serves as a high watermark(for breach of sla).

*{{slaDuration}}*
* I think we need an early alert level where we can alert the user in advance to attempt and
meet SLA. slaEnd is a high watermark only.

* if I use slaStart and slaEnd only, I don't have a low watermark(early alert to attempt to
meet sla) and a high watermark(indicate breach of sla). I can indirectly use slaStart and
slaDuration as a low watermark while slaEnd can potentially work as a high watermark. Slightly
convoluted but works.

* slaDuration is more flexible than another watermark on lines of slaEnd as it's relative
to actual time the process instance started and not to process instance nominal time. Having
a low watermark will trigger even in the case we crossed early alert time because input came
late. In this case both slaStart and low watermark alert will trigger. Having slaDuration
will avoid duplicate alerts in this case.

I will prefer to add slaDuration as well just for the use case of low watermark. If that can
be solved
otherwise then I am fine with skipping it. Let me know your thoughts. If we are not sure about
slaDuration we can start with slaStart and slaEnd only, instead of blocking the entire patch.



was (Author: ajayyadava):
I thought more on this and here are my thoughts on it:

*Burden for user?*
All the values are optional and user can configure only a few, so it shouldn't be a burden
on the user to configure a process. User can choose to not configure any SLA if he wishes
to do so.

*{{slaStart}}*
It is required to alert a user when the process hasn't kickstarted. 

*Won't sla for feeds(input feeds) solve for it?*
* sla for feeds is optional and hence it's not guaranteed that it will trigger.
* Configuring sla for one process is more convenient than configuring sla for all the feeds.
* It provides a very convenient reporting number to tell how many times the process input
was delayed. I can get process instance start times but I won't have a number to draw acceptable
boundaries. Both reporting dashboard and alerting software will be easy to keep consistent
if we use slaStart and read it from here. 
* What if it's a process with monthly frequency and someone puts an input feed's retention
not long enough. The feed won't alert in that case and process won't start either. I actually
witnessed it, a monthly process started at 6th of every month and used to consume a daily
feed for entire last month. Someone updated the daily feed's retention to 30 days and the
process missed SLA.

*{{slaEnd}}*
* I think slaEnd is primary component to monitor whether the process finished on time expected.
* Some processes may not have output feeds at all and hence it is important to have it.
This serves as a high watermark(for breach of sla).

*{{slaDuration}}*
* I think we need an early alert level where we can alert the user in advance to attempt and
meet SLA. slaEnd is a high watermark only.

* if I use slaStart and slaEnd only, I don't have a low watermark(early alert to attempt to
meet sla) and a high watermark(indicate breach of sla). I can indirectly use slaStart and
slaDuration as a low watermark while slaEnd can potentially work as a high watermark. Slightly
convoluted but works.

* slaDuration is more flexible than another watermark on lines of slaEnd as it's relative
to actual time the process started and not when process nominal time. Having a low watermark
will trigger even in the case we crossed early alert time because input came late. In this
case both slaStart and low watermark alert will trigger.


> Add SLA  for processes
> ----------------------
>
>                 Key: FALCON-722
>                 URL: https://issues.apache.org/jira/browse/FALCON-722
>             Project: Falcon
>          Issue Type: New Feature
>            Reporter: Ajay Yadav
>            Assignee: Ajay Yadav
>         Attachments: FALCON-722-v2.patch, FALCON-722.patch
>
>
> On lines of SLA for feeds, we want to have slaLow and slaHigh for processes as well.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message