samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yi Pan <nickpa...@gmail.com>
Subject Re: Question on zero downtime deployment of Samza jobs
Date Thu, 14 Jan 2016 18:14:14 GMT
Hi, Chinmay,

Did you forget to attach? I did not see the attachment in your last email.
Or is it due to the mail list restriction?

On Thu, Jan 14, 2016 at 10:12 AM, Chinmay Soman <chinmay.cerebro@gmail.com>
wrote:

> Sorry for the long delay (I was trying to find the design doc we made).
> Please see attachment for the design doc. I'm also CC'ing Peter Huang (the
> intern) who worked on this.
>
> Disclaimer: Given this was an internship project, we've cut some corners.
> We plan to come up with an improved version in some time.
>
> On Wed, Jan 6, 2016 at 11:31 AM, Yi Pan <nickpan47@gmail.com> wrote:
>
>> Hi, Chinmay,
>>
>> That's awesome! Could you share some design doc of this feature? We would
>> love to have this feature in LinkedIn as well!
>>
>> -Yi
>>
>> On Wed, Jan 6, 2016 at 10:02 AM, Chinmay Soman <chinmay.cerebro@gmail.com
>> >
>> wrote:
>>
>> > FYI: As part of an Uber internship project, we were working on exactly
>> this
>> > problem. Our approach was to do a rolling restart of all the containers
>> > wherein we start a "replica" container for each primary container and
>> let
>> > it "catch up" before we do the switch. Of course this doesn't guarantee
>> > zero downtime, but it does guarantee minimum time to upgrade each such
>> > container.
>> >
>> > The code is still in POC, but we do plan to finish this and make this
>> > available. Let me know if you're interested in trying it out.
>> >
>> > FYI: the sticky container deployment will also minimize the time to
>> upgrade
>> > / deploy since majority of the upgrade time is taken up by the
>> container in
>> > reading all the changelog (if any). Upgrade / re-deploy will also take a
>> > long time if the checkpoint topic is not log compacted (which is true in
>> > our environment).
>> >
>> > Thanks,
>> > C
>> >
>> > On Wed, Jan 6, 2016 at 9:56 AM, Bae, Jae Hyeon <metacret@gmail.com>
>> wrote:
>> >
>> > > Hi Samza devs and users
>> > >
>> > > I know this will be tricky in Samza because Samza Kafka consumer is
>> not
>> > > coordinated externally, but do you have any idea how to deploy samza
>> jobs
>> > > with zero downtime?
>> > >
>> > > Thank you
>> > > Best, Jae
>> > >
>> >
>> >
>> >
>> > --
>> > Thanks and regards
>> >
>> > Chinmay Soman
>> >
>>
>
>
>
> --
> Thanks and regards
>
> Chinmay Soman
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message