samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chinmay Soman <chinmay.cere...@gmail.com>
Subject Re: Question on zero downtime deployment of Samza jobs
Date Thu, 14 Jan 2016 18:15:34 GMT
It shows as attached in my sent email. That's weird.

On Thu, Jan 14, 2016 at 10:14 AM, Yi Pan <nickpan47@gmail.com> wrote:

> Hi, Chinmay,
>
> Did you forget to attach? I did not see the attachment in your last email.
> Or is it due to the mail list restriction?
>
> On Thu, Jan 14, 2016 at 10:12 AM, Chinmay Soman <chinmay.cerebro@gmail.com
> >
> wrote:
>
> > Sorry for the long delay (I was trying to find the design doc we made).
> > Please see attachment for the design doc. I'm also CC'ing Peter Huang
> (the
> > intern) who worked on this.
> >
> > Disclaimer: Given this was an internship project, we've cut some corners.
> > We plan to come up with an improved version in some time.
> >
> > On Wed, Jan 6, 2016 at 11:31 AM, Yi Pan <nickpan47@gmail.com> wrote:
> >
> >> Hi, Chinmay,
> >>
> >> That's awesome! Could you share some design doc of this feature? We
> would
> >> love to have this feature in LinkedIn as well!
> >>
> >> -Yi
> >>
> >> On Wed, Jan 6, 2016 at 10:02 AM, Chinmay Soman <
> chinmay.cerebro@gmail.com
> >> >
> >> wrote:
> >>
> >> > FYI: As part of an Uber internship project, we were working on exactly
> >> this
> >> > problem. Our approach was to do a rolling restart of all the
> containers
> >> > wherein we start a "replica" container for each primary container and
> >> let
> >> > it "catch up" before we do the switch. Of course this doesn't
> guarantee
> >> > zero downtime, but it does guarantee minimum time to upgrade each such
> >> > container.
> >> >
> >> > The code is still in POC, but we do plan to finish this and make this
> >> > available. Let me know if you're interested in trying it out.
> >> >
> >> > FYI: the sticky container deployment will also minimize the time to
> >> upgrade
> >> > / deploy since majority of the upgrade time is taken up by the
> >> container in
> >> > reading all the changelog (if any). Upgrade / re-deploy will also
> take a
> >> > long time if the checkpoint topic is not log compacted (which is true
> in
> >> > our environment).
> >> >
> >> > Thanks,
> >> > C
> >> >
> >> > On Wed, Jan 6, 2016 at 9:56 AM, Bae, Jae Hyeon <metacret@gmail.com>
> >> wrote:
> >> >
> >> > > Hi Samza devs and users
> >> > >
> >> > > I know this will be tricky in Samza because Samza Kafka consumer is
> >> not
> >> > > coordinated externally, but do you have any idea how to deploy samza
> >> jobs
> >> > > with zero downtime?
> >> > >
> >> > > Thank you
> >> > > Best, Jae
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Thanks and regards
> >> >
> >> > Chinmay Soman
> >> >
> >>
> >
> >
> >
> > --
> > Thanks and regards
> >
> > Chinmay Soman
> >
>



-- 
Thanks and regards

Chinmay Soman

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message