tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bikas Saha <bi...@apache.org>
Subject RE: api compatibility within a minor release
Date Thu, 10 Dec 2015 20:59:21 GMT
Yes. That is what I was trying to convey with a TezATSClient - it gives a Tez facade that provides
expected semantics like getFooForVertex() etc. and hides the actual ATS details from the user.

-----Original Message-----
From: Andre Kelpe [mailto:akelpe@concurrentinc.com] 
Sent: Thursday, December 10, 2015 2:09 AM
To: dev@tez.apache.org
Subject: Re: api compatibility within a minor release

>From our point of view ATS is an implementation detail. As you said in Budapest, it changes
all the time anyway, so it is not the API we would like to talk to. We care about the information
stored in the ATS that is relevant for the current DAG, but we don't really care about the
fact that the ATS is used. In MapReduce you also don't know, that the information is stored
int he JobHistoryServer. There is an API on the RunningJob class to get to that information.
We thing something along those lines would be good. Then you can change the DAGClient and
the storage for the information all you want, we as consumers never have to think about it.

- André

On Wed, Dec 9, 2015 at 11:39 PM, Bikas Saha <bikas@apache.org> wrote:

> Are we basically looking for detailed data that’s stored in the ATS? 
> Then should we consider creating a TezATSClient that understand Tez 
> semantics and fetches this information (in general for any DAG) 
> instead of augmenting DAGClient (that is responsible for submitting a 
> Tez DAG and monitoring a single DAG)?
>
> Bikas
>
> -----Original Message-----
> From: Andre Kelpe [mailto:akelpe@concurrentinc.com]
> Sent: Wednesday, December 9, 2015 10:32 AM
> To: Hitesh Shah <hitesh@apache.org>
> Cc: dev@tez.apache.org
> Subject: Re: api compatibility within a minor release
>
> Hi,
>
> (undusting this thread after a while)
>
> I have opened https://issues.apache.org/jira/browse/TEZ-2981 to work 
> around the issue. We believe that would be the best way forward.
>
> - André
>
> On Tue, Sep 15, 2015 at 11:04 PM, Hitesh Shah <hitesh@apache.org> wrote:
>
> > Hi Andre,
> >
> > I don’t believe I saw a JIRA created for this? Has this issue been 
> > resolved?
> >
> > Also, it would be good if you can collate any particular private 
> > APIs that you are using so that we can be aware of potential issues 
> > with such
> APIs.
> >
> > thanks
> > — Hitesh
> >
> > On Aug 20, 2015, at 2:10 AM, Andre Kelpe <akelpe@concurrentinc.com>
> wrote:
> >
> > > Thanks for the answer. We have a work-around for now. I am going 
> > > to make that inventory and submit a patch that changes the 
> > > annotations from @Private to @LimitedPrivate.
> > >
> > > From a semantic versioning point of view, I would still expect no
> > breaking
> > > changes in a bug fix release, but if the Tez community at large 
> > > can work with that, we have to accept that, I guess.
> > >
> > > - André
> > >
> > > On Tue, Aug 18, 2015 at 8:34 PM, Hitesh Shah <hitesh@apache.org>
> wrote:
> > >
> > >> Hello Andre,
> > >>
> > >> For the most part, @Private is considered internal implementation 
> > >> and subject to change at any point. In this case, even more so as 
> > >> it is an *Impl class.
> > >>
> > >> What we can do is try the following:
> > >>    - Look at all the various @Private classes being used by Cascading.
> > >>    - See which ones should not be used at all and which ones can 
> > >> be considered to be a @LimitedPrivate for Cascading.
> > >>    - For LimitedPrivate apis, we would then try to be a bit more 
> > >> careful with respect to changing/breaking these APIs. I would 
> > >> probably not say
> > that
> > >> they have will have the same guarantees as @Public/@Stable but we 
> > >> can
> > work
> > >> with the Cascading community to handle changes to these APIs in a
> > workable
> > >> manner on an ongoing basis.
> > >>
> > >> As for this API in question, for the short term fix, I guess a 
> > >> simple approach might be to introduce a backward compatible API ( 
> > >> I believe one which does not need the timeout param passed in )?
> > >> Would you mind
> > filing a
> > >> jira and hopefully provide a patch too?
> > >>
> > >> thanks
> > >> — Hitesh
> > >>
> > >> On Aug 18, 2015, at 8:01 AM, Andre Kelpe 
> > >> <akelpe@concurrentinc.com>
> > wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> I have found a small API incompatibility in the Tez 0.6.2 release.
> > >>> The DAGClientTimelineImpl constructor got a new parameter for
> time-outs.
> > This
> > >>> was not present in 0.6.1 and from a semantic versioning point of 
> > >>> view,
> > >> that
> > >>> is problematic. I know that the class is marked with the 
> > >>> @Private annotation, but it would be great if such incompatible 
> > >>> things aren't introduced in a bug-fix release. It would be 
> > >>> easier for downsteam
> > >> projects,
> > >>> if you just added a second constructor.
> > >>>
> > >>> Is that an oversight or are classes with the @Private annotation
> > subject
> > >> to
> > >>> change and dowstream projects simply have to deal with it?
> > >>>
> > >>> Thanks!
> > >>>
> > >>> - André
> > >>>
> > >>> --
> > >>> André Kelpe
> > >>> andre@concurrentinc.com
> > >>> http://concurrentinc.com
> > >>
> > >>
> > >
> > >
> > > --
> > > André Kelpe
> > > andre@concurrentinc.com
> > > http://concurrentinc.com
> >
> >
>
>
> --
> André Kelpe
> andre@concurrentinc.com
> http://concurrentinc.com
>
>


--
André Kelpe
andre@concurrentinc.com
http://concurrentinc.com


Mime
View raw message