samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yi Pan <nickpa...@gmail.com>
Subject Re: Question on zero downtime deployment of Samza jobs
Date Thu, 14 Jan 2016 18:17:35 GMT
It might be the mail list restriction. Could you try to my personal email
(i.e. nickpan47@gmail.com)?

On Thu, Jan 14, 2016 at 10:15 AM, Chinmay Soman <chinmay.cerebro@gmail.com>
wrote:

> It shows as attached in my sent email. That's weird.
>
> On Thu, Jan 14, 2016 at 10:14 AM, Yi Pan <nickpan47@gmail.com> wrote:
>
> > Hi, Chinmay,
> >
> > Did you forget to attach? I did not see the attachment in your last
> email.
> > Or is it due to the mail list restriction?
> >
> > On Thu, Jan 14, 2016 at 10:12 AM, Chinmay Soman <
> chinmay.cerebro@gmail.com
> > >
> > wrote:
> >
> > > Sorry for the long delay (I was trying to find the design doc we made).
> > > Please see attachment for the design doc. I'm also CC'ing Peter Huang
> > (the
> > > intern) who worked on this.
> > >
> > > Disclaimer: Given this was an internship project, we've cut some
> corners.
> > > We plan to come up with an improved version in some time.
> > >
> > > On Wed, Jan 6, 2016 at 11:31 AM, Yi Pan <nickpan47@gmail.com> wrote:
> > >
> > >> Hi, Chinmay,
> > >>
> > >> That's awesome! Could you share some design doc of this feature? We
> > would
> > >> love to have this feature in LinkedIn as well!
> > >>
> > >> -Yi
> > >>
> > >> On Wed, Jan 6, 2016 at 10:02 AM, Chinmay Soman <
> > chinmay.cerebro@gmail.com
> > >> >
> > >> wrote:
> > >>
> > >> > FYI: As part of an Uber internship project, we were working on
> exactly
> > >> this
> > >> > problem. Our approach was to do a rolling restart of all the
> > containers
> > >> > wherein we start a "replica" container for each primary container
> and
> > >> let
> > >> > it "catch up" before we do the switch. Of course this doesn't
> > guarantee
> > >> > zero downtime, but it does guarantee minimum time to upgrade each
> such
> > >> > container.
> > >> >
> > >> > The code is still in POC, but we do plan to finish this and make
> this
> > >> > available. Let me know if you're interested in trying it out.
> > >> >
> > >> > FYI: the sticky container deployment will also minimize the time to
> > >> upgrade
> > >> > / deploy since majority of the upgrade time is taken up by the
> > >> container in
> > >> > reading all the changelog (if any). Upgrade / re-deploy will also
> > take a
> > >> > long time if the checkpoint topic is not log compacted (which is
> true
> > in
> > >> > our environment).
> > >> >
> > >> > Thanks,
> > >> > C
> > >> >
> > >> > On Wed, Jan 6, 2016 at 9:56 AM, Bae, Jae Hyeon <metacret@gmail.com>
> > >> wrote:
> > >> >
> > >> > > Hi Samza devs and users
> > >> > >
> > >> > > I know this will be tricky in Samza because Samza Kafka consumer
> is
> > >> not
> > >> > > coordinated externally, but do you have any idea how to deploy
> samza
> > >> jobs
> > >> > > with zero downtime?
> > >> > >
> > >> > > Thank you
> > >> > > Best, Jae
> > >> > >
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Thanks and regards
> > >> >
> > >> > Chinmay Soman
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Thanks and regards
> > >
> > > Chinmay Soman
> > >
> >
>
>
>
> --
> Thanks and regards
>
> Chinmay Soman
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message