samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chinmay Soman <chinmay.cere...@gmail.com>
Subject Re: Question on zero downtime deployment of Samza jobs
Date Thu, 14 Jan 2016 18:24:52 GMT
FYI: I've forwarded it to Yi's personal email. Yi: please feel free to
email Peter (CC'ed) or me about any questions. FYI: the real solution would
be to implement standby containers. This solution is an attempt to do the
same.

On Thu, Jan 14, 2016 at 10:17 AM, Yi Pan <nickpan47@gmail.com> wrote:

> It might be the mail list restriction. Could you try to my personal email
> (i.e. nickpan47@gmail.com)?
>
> On Thu, Jan 14, 2016 at 10:15 AM, Chinmay Soman <chinmay.cerebro@gmail.com
> >
> wrote:
>
> > It shows as attached in my sent email. That's weird.
> >
> > On Thu, Jan 14, 2016 at 10:14 AM, Yi Pan <nickpan47@gmail.com> wrote:
> >
> > > Hi, Chinmay,
> > >
> > > Did you forget to attach? I did not see the attachment in your last
> > email.
> > > Or is it due to the mail list restriction?
> > >
> > > On Thu, Jan 14, 2016 at 10:12 AM, Chinmay Soman <
> > chinmay.cerebro@gmail.com
> > > >
> > > wrote:
> > >
> > > > Sorry for the long delay (I was trying to find the design doc we
> made).
> > > > Please see attachment for the design doc. I'm also CC'ing Peter Huang
> > > (the
> > > > intern) who worked on this.
> > > >
> > > > Disclaimer: Given this was an internship project, we've cut some
> > corners.
> > > > We plan to come up with an improved version in some time.
> > > >
> > > > On Wed, Jan 6, 2016 at 11:31 AM, Yi Pan <nickpan47@gmail.com> wrote:
> > > >
> > > >> Hi, Chinmay,
> > > >>
> > > >> That's awesome! Could you share some design doc of this feature? We
> > > would
> > > >> love to have this feature in LinkedIn as well!
> > > >>
> > > >> -Yi
> > > >>
> > > >> On Wed, Jan 6, 2016 at 10:02 AM, Chinmay Soman <
> > > chinmay.cerebro@gmail.com
> > > >> >
> > > >> wrote:
> > > >>
> > > >> > FYI: As part of an Uber internship project, we were working on
> > exactly
> > > >> this
> > > >> > problem. Our approach was to do a rolling restart of all the
> > > containers
> > > >> > wherein we start a "replica" container for each primary container
> > and
> > > >> let
> > > >> > it "catch up" before we do the switch. Of course this doesn't
> > > guarantee
> > > >> > zero downtime, but it does guarantee minimum time to upgrade
each
> > such
> > > >> > container.
> > > >> >
> > > >> > The code is still in POC, but we do plan to finish this and make
> > this
> > > >> > available. Let me know if you're interested in trying it out.
> > > >> >
> > > >> > FYI: the sticky container deployment will also minimize the time
> to
> > > >> upgrade
> > > >> > / deploy since majority of the upgrade time is taken up by the
> > > >> container in
> > > >> > reading all the changelog (if any). Upgrade / re-deploy will
also
> > > take a
> > > >> > long time if the checkpoint topic is not log compacted (which
is
> > true
> > > in
> > > >> > our environment).
> > > >> >
> > > >> > Thanks,
> > > >> > C
> > > >> >
> > > >> > On Wed, Jan 6, 2016 at 9:56 AM, Bae, Jae Hyeon <
> metacret@gmail.com>
> > > >> wrote:
> > > >> >
> > > >> > > Hi Samza devs and users
> > > >> > >
> > > >> > > I know this will be tricky in Samza because Samza Kafka
consumer
> > is
> > > >> not
> > > >> > > coordinated externally, but do you have any idea how to
deploy
> > samza
> > > >> jobs
> > > >> > > with zero downtime?
> > > >> > >
> > > >> > > Thank you
> > > >> > > Best, Jae
> > > >> > >
> > > >> >
> > > >> >
> > > >> >
> > > >> > --
> > > >> > Thanks and regards
> > > >> >
> > > >> > Chinmay Soman
> > > >> >
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks and regards
> > > >
> > > > Chinmay Soman
> > > >
> > >
> >
> >
> >
> > --
> > Thanks and regards
> >
> > Chinmay Soman
> >
>



-- 
Thanks and regards

Chinmay Soman

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message