aries-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Francois Papon <francois.pa...@openobject.fr>
Subject Re: [Proposal] Create new sub project for journaled messaging
Date Wed, 02 Jan 2019 12:17:03 GMT
Hi,

This thread is very interesting and I would be happy to help for testing
the integration with Karaf Decanter :)

Regards,

François Papon
fpapon@apache.org

Le 27/12/2018 à 21:42, Christian Schneider a écrit :
> Your help is more than welcome.
>
> I will start a new thread to discuss the interface and adapters/impls.
>
> Christian
>
> Am Sa., 22. Dez. 2018 um 15:11 Uhr schrieb Jean-Baptiste Onofré <
> jb@nanthrax.net>:
>
>> I see, thanks for the details.
>>
>> If you think there's several use cases and adoption, I agree with a new
>> sub-project and spec.
>>
>> I would be more than happy to help, I'm working a lot with Kafka and
>> Zookeeper recently (especially new features coming in ActiveMQ).
>>
>> Regards
>> JB
>>
>> On 22/12/2018 14:46, Christian Schneider wrote:
>>> Decanter is based in EventAdmin. I like the nice decoupling EventAdmin
>>> gives us for decantr but it is limited by EventAdmin's deficiencies.
>>> For example any Events sent before a subscriber arrives are lost.
>>> The main advantage of the API compared to simple pub/sub is that you can
>> go
>>> back in the history (like in kafka).
>>>
>>> So I think either the API directly or an EventAdmin based on the jounaled
>>> messaging API could be very valuable for Decanter to make sure no startup
>>> messages are lost.
>>>
>>> Christian
>>>
>>> Am Sa., 22. Dez. 2018 um 14:34 Uhr schrieb Jean-Baptiste Onofré <
>>> jb@nanthrax.net>:
>>>
>>>> OK, it's clear now.
>>>>
>>>> It sounds a bit like Decanter (EventAdmin sent to a storage backend),
>> no ?
>>>> Regards
>>>> JB
>>>>
>>>> On 22/12/2018 14:30, Christian Schneider wrote:
>>>>> Hi JB,
>>>>>
>>>>> the idea is not to replace kafka. Instead we want to provide an API
>> that
>>>>> can abstract kafka as well as other journal based pub sub systems.
>>>>> So actually one implementation of the api would bridge to kafka. We
>> also
>>>>> plan a mongo and a in memory impl.
>>>>>
>>>>> One use case for the api would be a new EventAdmin with history.
>> Another
>>>> is
>>>>> logging that ensures no messages are lost at startup even if the real
>>>>> backend appears later.
>>>>>
>>>>> Christian
>>>>>
>>>>> Am Sa., 22. Dez. 2018 um 13:13 Uhr schrieb Jean-Baptiste Onofré <
>>>>> jb@nanthrax.net>:
>>>>>
>>>>>> Hi Christian,
>>>>>>
>>>>>> I'm not sure I got the scope.
>>>>>>
>>>>>> Is it a new pub/sub system like Kafka or Google PubSub (or even
>>>>>> ActiveMQ/AmazonMQ) ?
>>>>>> Is it something like eventadmin on cloud (like we do in Cellar with
>>>>>> Hazelcast) ?
>>>>>>
>>>>>> If it's just a pure pub/sub, I don't see a huge value compared to
>> Kafka.
>>>>>> About a spec, if we only have one impl of the spec, and the spec
is
>>>>>> mainly "focused" on the current impl, not sure it makes sense as
well.
>>>>>>
>>>>>> Thoughts ?
>>>>>>
>>>>>> Regards
>>>>>> JB
>>>>>>
>>>>>> On 22/12/2018 09:38, Christian Schneider wrote:
>>>>>>> Some Background
>>>>>>>
>>>>>>> At Adobe I and two colleagues Timothee Maret and Alexei Krainiouk
>> work
>>>> on
>>>>>>> making the AEM content publishing fit for the cloud. It is about
>>>>>>> transporting content from author instances that are only visible
to a
>>>>>>> customer to outside facing instances that handle the load. For
this
>>>>>> effort
>>>>>>> it is important to have a reliable pub/sub messaging with support
of
>> a
>>>>>>> history where each consumer can decide where to start consuming
from.
>>>>>>>
>>>>>>> We found that Apache Kafka is a good fit function wise but a
little
>>>> heavy
>>>>>>> regarding deployment and management. So to be more versatile
we
>> thought
>>>>>>> about encapsulating the messaging using an API and providing
>>>>>>> implementations for different backends.
>>>>>>> Unfortunately there is no existing API that solves this.
>>>>>>>
>>>>>>> Use case and API sketch
>>>>>>>
>>>>>>> So we created a description of the use cases as well as a sketch
for
>> a
>>>>>>> minimal API.
>>>>>>> See
>>>> https://github.com/cschneider/journaled-events/blob/master/README.md
>>>>>>> Proposal
>>>>>>>
>>>>>>> We would like to develop this API as well as implementations
as a sub
>>>>>>> project at Apache Aries.
>>>>>>> The main reason for choosing Aries is that David told us that
there
>> is
>>>>>>> interest in a OSGi spec about a similar purpose. So we might
also
>> bring
>>>>>>> this into an OSGi spec.
>>>>>>> Alexei and Timothee are not yet committers at Aries. I think
we can
>>>> work
>>>>>>> with them using the normal contribution model for the start and
make
>>>> them
>>>>>>> committers after a few PRs.
>>>>>>>
>>>>>>> So what do you think?
>>>>>>> As a first step I would like to discuss if the Aries PMC is
>> interested
>>>> in
>>>>>>> giving this effort a home.  After that we can discuss the actual
API
>> as
>>>>>>> well as possible usages and backends.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Christian
>>>>>>>
>>>>>> --
>>>>>> Jean-Baptiste Onofré
>>>>>> jbonofre@apache.org
>>>>>> http://blog.nanthrax.net
>>>>>> Talend - http://www.talend.com
>>>>>>
>>>>>
>>>> --
>>>> Jean-Baptiste Onofré
>>>> jbonofre@apache.org
>>>> http://blog.nanthrax.net
>>>> Talend - http://www.talend.com
>>>>
>>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>

Mime
View raw message