synapse-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shafreen <anfar.shafr...@gmail.com>
Subject Re: Message Forwarding Processor consumes all the messages at once
Date Sat, 25 Jul 2020 16:47:24 GMT
Hi All,

Please find the documentation PR [1] for this feature.

[1] https://github.com/apache/synapse/pull/47

On Sat, Jul 25, 2020 at 11:13 AM Shafreen <anfar.shafreen@gmail.com> wrote:

> Hi All,
>
> Can someone kindly look into the aforementioned PR and merge it. I will
> send the documentation PR soon after that.
>
> On Sat, Jul 11, 2020 at 10:34 PM Shafreen <anfar.shafreen@gmail.com>
> wrote:
>
>> Hi All,
>>
>> Please find the code change for this feature in PR [1].
>>
>> [1] https://github.com/apache/synapse/pull/46
>>
>> On Mon, Jul 6, 2020 at 11:23 PM Vanjikumaran Sivajothy <
>> vanjikumaran@gmail.com> wrote:
>>
>>> last suggestion much more intuitive.
>>>
>>> On Mon, Jul 6, 2020 at 9:27 AM Shafreen <anfar.shafreen@gmail.com>
>>> wrote:
>>>
>>>> On Sun, Jun 28, 2020 at 11:39 PM Vanjikumaran Sivajothy <
>>>> vanjikumaran@gmail.com> wrote:
>>>>
>>>>> Since it is a part of the Message processor itself;
>>>>> Why can't we stick to  <parameter name="throttle">true</parameter>?
>>>>>
>>>>
>>>> Yeah. I think you are right. Compared to what I suggested *throttle* seems
>>>> to be better.
>>>>
>>>> <messageProcessor class=
>>>> "org.apache.synapse.message.processors.forward.ScheduledMessageForwardingProcessor"
>>>> name="ScheduledProcessor" messageStore="MyStore"> <parameter name=
>>>> "interval">10000</parameter> *<parameter
>>>> name="throttle">true</parameter>* <parameter name=
>>>> "max.deliver.attempts">3</parameter> <parameter name="max.deliver.drop"
>>>> >true</parameter> </messageProcessor>
>>>>
>>>> Other option we can go with is *consume.all *and the configuration
>>>> would look like as below,
>>>>
>>>> <messageProcessor class=
>>>> "org.apache.synapse.message.processors.forward.ScheduledMessageForwardingProcessor"
>>>> name="ScheduledProcessor" messageStore="MyStore"> <parameter name=
>>>> "interval">10000</parameter> *<parameter
>>>> name="consume.all">true</parameter>* <parameter name=
>>>> "max.deliver.attempts">3</parameter> <parameter name="max.deliver.drop"
>>>> >true</parameter> </messageProcessor>
>>>>
>>>> I prefer the latter but what do you think ?
>>>>
>>>>
>>>>> On Sun, Jun 28, 2020 at 10:32 AM Shafreen <anfar.shafreen@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> On Sun, Jun 14, 2020 at 8:15 AM Shafreen <anfar.shafreen@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> If everyone is okay. I can go ahead with the implementation.
The
>>>>>>> below is the property that will be introduced.
>>>>>>>
>>>>>>> <parameter name="throttle.message.processing">true</parameter>
>>>>>>>
>>>>>>> Users can use the property to decide what they want to do when
the
>>>>>>> message processor is triggered.
>>>>>>>
>>>>>>> 1. Consume all the messages at once
>>>>>>> 2. Consume messages at the rate in which message processor is
>>>>>>> triggered
>>>>>>>
>>>>>>
>>>>>> Any idea on the aforementioned property ? If you all are okay I can
>>>>>> go ahead and implement the feature.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Shafreen
>>>>>>>
>>>>>>> On Fri, May 29, 2020 at 7:18 PM Shafreen <anfar.shafreen@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I thought of simply using the time interval that is there
for
>>>>>>>> Message Processor.
>>>>>>>>
>>>>>>>> On Fri, May 29, 2020 at 12:04 PM Vanjikumaran Sivajothy <
>>>>>>>> vanjikumaran@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> What about the configuration of the interval?
>>>>>>>>>
>>>>>>>>> On Thu, May 28, 2020 at 8:53 AM Shafreen <anfar.shafreen@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi All,
>>>>>>>>>>
>>>>>>>>>> How about a property name as below? we can make it
false by
>>>>>>>>>> default which would give us the current behaviour.
>>>>>>>>>>
>>>>>>>>>> <parameter name="throttle.message.processing">true</parameter>
>>>>>>>>>>
>>>>>>>>>> On Mon, May 25, 2020 at 10:41 AM prabath <prabathmail@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> +1 to support both behaviors.
>>>>>>>>>>>
>>>>>>>>>>> On Sun, May 24, 2020 at 12:26 AM Shafreen <
>>>>>>>>>>> anfar.shafreen@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Vanji,
>>>>>>>>>>>>
>>>>>>>>>>>> On Sat, May 23, 2020 at 2:51 PM Vanjikumaran
Sivajothy <
>>>>>>>>>>>> vanjikumaran@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Let’s consider the current behaviors
as a default.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Adding additional Optional property may
control to make sure
>>>>>>>>>>>>> the backward comparability.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Okay. We can add a new parameter, that way
we can have both
>>>>>>>>>>>> behaviors as you mentioned.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sat, May 23, 2020 at 12:46 AM Shafreen
<
>>>>>>>>>>>>> anfar.shafreen@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Current behaviour of the Message
Forwarding Processor is to
>>>>>>>>>>>>>> consume all the messages at once.
For instance, say, the Message Forwarding
>>>>>>>>>>>>>> Processor is configured to run every
10 seconds and the Message store is
>>>>>>>>>>>>>> filled with 5 messages within the
10 second gap. In such a situation,
>>>>>>>>>>>>>> Message Forwarding Processor consumes
all 5 messages and try to send it to
>>>>>>>>>>>>>> back-end as fast as possible. I think
this behaviour is not optimal. The
>>>>>>>>>>>>>> purpose of Message Forwarding Processor
it to send messages to the back-end
>>>>>>>>>>>>>> in a controlled rate. So that the
back-end server can handle the load. IMO,
>>>>>>>>>>>>>> ideal behaviour should be to consume
one message at a time and try to send
>>>>>>>>>>>>>> it to the back-end as per the configured
interval.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> However, if the configured interval
is a cron expression, in
>>>>>>>>>>>>>> such cases the current behaviour
is correct. Because cron expressions could
>>>>>>>>>>>>>> have very large intervals.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Therefore, I think it is best to
keep the current behaviour
>>>>>>>>>>>>>> for cron intervals but change it
for normal intervals as aforementioned.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> WDYT ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Shafreen
>>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Sent from Gmail Mobile
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Prabath Ariyarathna.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Best Regards,
>>>>>>>>> Vanji
>>>>>>>>>
>>>>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>> Vanji
>>>>>
>>>>
>>>
>>> --
>>> Best Regards,
>>> Vanji
>>>
>>

Mime
View raw message