hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "J. Rottinghuis" <jrottingh...@gmail.com>
Subject Re: [VOTE] Merge feature branch YARN-5355 (Timeline Service v2) to trunk
Date Sun, 27 Aug 2017 18:01:38 GMT
That makes sense to me, after merge to trunk and branch 2, it probably
makes sense to create a new jira and feature branch for atsv2 additional
feature dev.

Cheers,

Joep

On Fri, Aug 25, 2017 at 3:05 PM, Vrushali C <vrushalic2016@gmail.com> wrote:

> Hi Subru,
>
> Thanks for your vote and your response!
>
> Regarding your question about merging to branch2, I think, after the merge
> to trunk is done, it will be good to merge to branch-2 as soon as we can.
> A lot of testing with trunk has been done presently and it will be good to
> repeat the same with the YARN-5355_branch2 branch before we start working
> on newer features. That way trunk and branch-2 would be in a similar state
> with respect to timeline service v2 development.
>
> thanks
> Vrushali
>
>
>
>
>
>
>
> On Fri, Aug 25, 2017 at 2:28 PM, varunsaxena@apache.org <
> varun.saxena.apache@gmail.com> wrote:
>
> > Thanks Subru for voting.
> >
> >
> >> What are the timelines you are looking for getting this into branch-2?
> >
> > We haven't yet decided on it and were thinking of discussing this in
> > detail within the team after merge to trunk.
> > The timelines would depend on whether we release whatever we merge to
> > trunk in 2.9 or would we want to get in few other features which people
> > would like to see in 2.9
> > This would require some discussion with the stakeholders.
> > We were thinking of having a short discussion with you guys as well to
> > find out whether there are any further gaps in ATSv2 with respect to
> > federation support and if they can be filled before 2.9 release.
> >
> > Assuming 2.9 is targetted for October end, we would have to start a merge
> > vote in September end or October 1st week which leaves us with very
> little
> > time to take up large changes anyways.
> >
> > We do maintain a branch-2 version of ATSv2(YARN-5355_branch2) though
> which
> > we rebase with branch-2 regularly. So, if we decide to merge in branch-2
> > without any additional changes, we would be able to go for branch-2 merge
> > discussion and vote almost immediately.
> >
> > Regards,
> > Varun Saxena.
> >
> >
> >
> >
> > On Sat, Aug 26, 2017 at 2:00 AM, Subramaniam V K <subru.vk@gmail.com>
> > wrote:
> >
> >> +1 (binding).
> >>
> >> I have been following the effort and had few design discussions around
> the
> >> team especially about how it integrates with Federation. Overall I feel
> >> it's a welcome improvement to YARN.
> >>
> >> What are the timelines you are looking for getting this into branch-2?
> >>
> >> Thanks,
> >> Subru
> >>
> >> On Fri, Aug 25, 2017 at 10:04 AM, Sangjin Lee <sjlee@apache.org> wrote:
> >>
> >> > +1 (binding)
> >> >
> >> > I've built the current branch, and checked out a few basic areas
> >> including
> >> > documentation. Also perused the most recent changes that went in.
> >> >
> >> > Thanks much for the great team work! I look forward to seeing it in
> >> action.
> >> >
> >> > Regards,
> >> > Sangjin
> >> >
> >> > On Fri, Aug 25, 2017 at 9:27 AM, Haibo Chen <haibochen@cloudera.com>
> >> > wrote:
> >> >
> >> > > +1 from my side.
> >> > >
> >> > > More from the perspective of ensuring there is no impact of ATSv2
> >> when it
> >> > > is off (by default), I deployed the latest YARN-5355 bits into a few
> >> > > clusters and ran internal Smoke tests. The tests shows no impact
> when
> >> > ATSv2
> >> > > is off.
> >> > >
> >> > > Best,
> >> > > Haibo
> >> > >
> >> > > On Thu, Aug 24, 2017 at 7:51 AM, Sunil G <sunilg@apache.org>
wrote:
> >> > >
> >> > > > Thank you very much Vrushali, Rohith, Varun and other folks who
> made
> >> > this
> >> > > > happen. Great work, really appreciate the same!!
> >> > > >
> >> > > > +1 (binding) from my side:
> >> > > >
> >> > > > # Tested ATSv2 cluster in a secure cluster. Ran some basic jobs
> >> > > > # Accessed new YARN UI which shows various flows/flow activity
> etc.
> >> > Seems
> >> > > > fine.
> >> > > > # Based on code, looks like all apis are compatible.
> >> > > > # REST api docs looks fine as well, I guess we could improve
that
> a
> >> bit
> >> > > > more post merge as well.
> >> > > > # Adding to additional thoughts which are discussed here, native
> >> > service
> >> > > > also could publish events to atsv2. I think that work is also
> >> happened
> >> > in
> >> > > > branch.
> >> > > >
> >> > > > Looking forward to a much wider adoption of ATSv2 with more
> >> projects.
> >> > > >
> >> > > > Thanks
> >> > > > Sunil
> >> > > >
> >> > > >
> >> > > > On Tue, Aug 22, 2017 at 12:02 PM Vrushali Channapattan <
> >> > > > vrushalic2016@gmail.com> wrote:
> >> > > >
> >> > > > > Hi folks,
> >> > > > >
> >> > > > > Per earlier discussion [1], I'd like to start a formal vote
to
> >> merge
> >> > > > > feature branch YARN-5355 [2] (Timeline Service v.2) to trunk.
> The
> >> > vote
> >> > > > will
> >> > > > > run for 7 days, and will end August 29 11:00 PM PDT.
> >> > > > >
> >> > > > > We have previously completed one merge onto trunk [3] and
> Timeline
> >> > > > Service
> >> > > > > v2 has been part of Hadoop release 3.0.0-alpha1.
> >> > > > >
> >> > > > > Since then, we have been working on extending the capabilities
> of
> >> > > > Timeline
> >> > > > > Service v2 in a feature branch [2] for a while, and we are
> >> reasonably
> >> > > > > confident that the state of the feature meets the criteria
to be
> >> > merged
> >> > > > > onto trunk and we'd love folks to get their hands on it
in a
> test
> >> > > > capacity
> >> > > > > and provide valuable feedback so that we can make it
> >> > production-ready.
> >> > > > >
> >> > > > > In a nutshell, Timeline Service v.2 delivers significant
> >> scalability
> >> > > and
> >> > > > > usability improvements based on a new architecture. What
we
> would
> >> > like
> >> > > to
> >> > > > > merge to trunk is termed "alpha 2" (milestone 2). The feature
> has
> >> a
> >> > > > > complete end-to-end read/write flow with security and read
level
> >> > > > > authorization via whitelists. You should be able to start
> setting
> >> it
> >> > up
> >> > > > and
> >> > > > > testing it.
> >> > > > >
> >> > > > > At a high level, the following are the key features that
have
> been
> >> > > > > implemented since alpha1:
> >> > > > > - Security via Kerberos Authentication and delegation tokens
> >> > > > > - Read side simple authorization via whitelist
> >> > > > > - Client configurable entity sort ordering
> >> > > > > - Richer REST APIs for apps, app attempts, containers, fetching
> >> > metrics
> >> > > > by
> >> > > > > timerange, pagination, sub-app entities
> >> > > > > - Support for storing sub-application entities (entities
that
> >> exist
> >> > > > outside
> >> > > > > the scope of an application)
> >> > > > > - Configurable TTLs (time-to-live) for tables, configurable
> table
> >> > > > prefixes,
> >> > > > > configurable hbase cluster
> >> > > > > - Flow level aggregations done as dynamic (table level)
> >> coprocessors
> >> > > > > - Uses latest stable HBase release 1.2.6
> >> > > > >
> >> > > > > There are a total of 82 subtasks that were completed as
part of
> >> this
> >> > > > > effort.
> >> > > > >
> >> > > > > We paid close attention to ensure that once disabled Timeline
> >> Service
> >> > > v.2
> >> > > > > does not impact existing functionality when disabled (by
> default).
> >> > > > >
> >> > > > > Special thanks to a team of folks who worked hard and
> contributed
> >> > > towards
> >> > > > > this effort with patches, reviews and guidance: Rohith Sharma
K
> S,
> >> > > Varun
> >> > > > > Saxena, Haibo Chen, Sangjin Lee, Li Lu, Vinod Kumar Vavilapalli,
> >> Joep
> >> > > > > Rottinghuis, Jason Lowe, Jian He, Robert Kanter, Micheal
Stack.
> >> > > > >
> >> > > > > Regards,
> >> > > > > Vrushali
> >> > > > >
> >> > > > > [1] http://www.mail-archive.com/yarn-dev@hadoop.apache.org/
> msg27
> >> > > 383.html
> >> > > > > [2] https://issues.apache.org/jira/browse/YARN-5355
> >> > > > > [3] https://issues.apache.org/jira/browse/YARN-2928
> >> > > > > [4] https://github.com/apache/hadoop/commits/YARN-5355
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message