jclouds-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ignasi Barrera <n...@apache.org>
Subject RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?
Date Thu, 18 Dec 2014 08:46:54 GMT
Right now, this procedure is completely irrelevant for this discussion: do
you know right now how many labs repos would we have? No.

Also that procedure could seem a long one but all those actions are likely
to happen very separated in time. It will take time since an
api/abstraction starts until it is in a good shape to be promoted, so I
wouldn't take that procedure as a step by step one to follow. How often do
we promote apis or providers? Do we really need a such a step by step
procedure? We have done this many times in the past and there was no issue
about how to do it.

I think that the peocedure (which I also consider unnecessary) is
irrelevant for this discussion. Let's focus on the subject of the thread
and elaborate on that.
El 16/12/2014 20:50, "Zack Shoylev" <zack.shoylev@rackspace.com> escribió:

> Let's work out some examples about promoting APIs and how the repos should
> work.
>
> Example: I have a provider-specific API that extends the compute
> abstraction in a meaningful way.
> Currently I would have to:
> 1. Make a PR for the API in one of the labs
> 2. Make a PR for the abstraction in jclouds/jclouds
> 3. (Same as 1. in a way) Make a PR in other labs repos for other providers
> (as it is likely new abstraction features would have to be supported in
> other providers - some of these might be created by other people, obviously
> - or these features might already be supported)
> 4. Review and merge all of these.
> 5. Make PRs to promote the provider-specific APIs to jclouds/jclouds and
> merge those.
> 6. Make PRs to clean up labs repos.
> Does this make sense? Am I missing something? How can we improve this?
>
> ________________________________________
> From: Ignasi Barrera [nacx@apache.org]
> Sent: Monday, December 15, 2014 3:22 PM
> To: dev@jclouds.apache.org
> Subject: Re: [DISCUSS] What is the criteria for something to be in the
> jclouds/jclouds repo?
>
> I can agree that giving full control over provider specific APIs can
> be a good thing, but always as a secondary goal. As you say,
> abstractions and working on the missing ones should be the high
> priority. Adding provider specific features to complete the
> abstractions is also a common sense thing; and at this concrete point
> in time, keep growing or promoting the APIs that are not related to
> any abstraction don't benefit the project as a whole.
>
> I think the entire point of this discussion is to set up a baseline
> that allows us to "move forward". To do that, we need to have a common
> criteria and a common vision of which should be the next steps to
> evolve the project, and it seems we agree that the high priority
> should be to work on the abstractions and focus on them.
>
> This means that specially the PMC and committers should work to make
> this happen, and focus on this high priority tasks. Setting a rule
> that we need an abstraction to promote the APIs will help us have
> under control what we promote and help us focus on the tasks we have
> defined as high priority. If we keep putting our major efforts in
> things that are not the high priority (apis not related to an
> abstraction), we won't achieve the main purpose of "moving forward".
>
> This is not black or white, and I don't think "we will never promote
> an API that doesn't have an abstraction" is an statement we should do.
> However, at this point in time, we should *really enforce* any
> promoted provider and API to fit in an abstraction. We have limited
> resources, and we need to focus in the high priority tasks. If we
> don't dedicate these few resources to the high priority tasks, we
> won't succeed.
>
>
> So, "will having an abstraction be a mandatory requirement forever and
> ever?" I'd say No.
> But that will only make sense if we work *starting today* together on
> the APIs that actually fit on those abstractions, and start building
> and discussing the ones that could be added. We should prioritise
> that, and make that a *real* thing, leaving the unrelated
> apis/providers aside. Otherwise we will be at the same point and won't
> be able to move forward, regardless of what we promote.
>
>
>
>
>
> On 15 December 2014 at 18:49, Zack Shoylev <zack.shoylev@rackspace.com>
> wrote:
> >>> Every single API in jclouds must be aligned with the project's
> purpose, and
> > that means *portability*.
> >
> > jclouds also tries to do portability "while giving you full control to
> use cloud-specific features."
> >
> >>>  if someone contributes a patch adding support for Google Drive
> >
> > I was not part of that discussion, but it should be possible to add a
> blobstore abstraction for this, no?
> >
> >>> So, once again, jclouds is *not* a cloud aggregator for any API
> labelled
> > "cloud". It is about portable code, and every single API and provider
> > should meet the project goals.
> >
> > jclouds allows you to write portable applications, but also allows you
> the flexibility to use cloud-specific features. From a user perspective,
> you need to have both. You want to be able to build infrastructure using
> high-level code that works with all the clouds, but occasionally you will
> need the flexibility to do provider-specific things (even just because this
> is how providers differentiate themselves).
> >
> >
> > I wanted to focus on the user perspective for this discussion. In the
> end, users want things to work well, with less code, and with whatever
> providers they choose.
> > Because jclouds is modular, users don't care if jclouds "is not an API
> aggregator of any API that is labelled "cloud"." - users want jclouds to
> work well for whatever providers and clouds they choose.
> > As such, figuring out the criteria of what should be in jclouds/jclouds
> is not really something that affects users directly, but has to be done to
> make development easier, saner, and to reduce current and future technical
> debt.
> > If we had infinite resources, it would make sense, for users, to include
> and release all cloud APIs and their abstractions (where such abstractions
> make sense) as part of jclouds.
> >
> > Part of this is about balance. Currently we have too many cloud-specific
> APIs that could be abstracted, but are not. Clearly some of the
> high-priority work in jclouds is working on abstractions. That's the case
> at the moment. It might be a knee-jerk reaction to limit jclouds just to
> abstractable APIs because of this situation, when in the future we might
> also want to give users the flexibility to also use provider-specific
> features or even provider-specific APIs that might be useful when used
> together with higher-level, more abstract code.
> >
> > This is our current mission statement:
> > "Apache jclouds® is an open source multi-cloud toolkit for the Java
> platform that gives you the freedom to create applications that are
> portable across clouds while giving you full control to use cloud-specific
> features."
> >
> > Should this be morphed into:
> > "Apache jclouds® is an open source multi-cloud toolkit for the Java
> platform that gives you the freedom to create applications that are
> portable across clouds"? Is this better for users?
> >
> > Thanks, and let's keep the discussion going. Figuring things out will
> help both contributors and users.
> >
> > ________________________________________
> > From: Ignasi Barrera [nacx@apache.org]
> > Sent: Wednesday, December 10, 2014 12:59 AM
> > To: dev@jclouds.apache.org
> > Subject: RE: [DISCUSS] What is the criteria for something to be in the
> jclouds/jclouds repo?
> >
> > I'd never do point 2 and would remove those APIs. As I've said several
> > times in the thread, jclouds is not an API aggregator of any API that is
> > labelled "cloud".
> >
> > Every single API in jclouds must be aligned with the project's purpose,
> and
> > that means *portability*. It does not make sense to keep adding APIs that
> > don't fit in the project's purpose. If we do that we would be facing the
> > exact same issues we are facing now (I mentioned them in my previous
> email)
> > regardless of the number of repos we use.
> >
> > To put a concrete example: jclouds is *not* the Rackspace SDK nor is the
> > OpenStack java SDK. Perhaps Rackspace adopts jclouds as its SDK, but
> > jclouds isn't the Rackspace SDK. And this difference is important.
> Perhaps
> > not every single API in Rackspace makes sense in jclouds, as it would
> > hardly be modelled in a portable way. In that case, we shouldn't add it,
> as
> > it does not benefit jclouds as a whole and doesn't serve the jclouds
> > purpose.
> >
> > Let's take another example: if someone contributes a patch adding support
> > for Google Drive (which was discussed in the ML or IRC), providing an API
> > to store documents, should we merge it? That API does not fit in the
> > BlobStore abstraction as it does not manage blobs (raw object data) but
> > documents. The answer is no, because jclouds is not about that (ir is
> just
> > a single, isolated API to store documents out there). Without alignment
> > with what jclouds targets, we shouldn't add it.
> >
> > So, once again, jclouds is *not* a cloud aggregator for any API labelled
> > "cloud". It is about portable code, and every single API and provider
> > should meet the project goals. If not, we should not add them.
> > El 10/12/2014 01:54, "Zack Shoylev" <zack.shoylev@rackspace.com>
> escribió:
> >
> >> If we take this to its logical conclusion, it sounds like we can
> separate
> >> APIs into two categories:
> >>
> >> 1. Those that have an abstraction or can be abstracted in the future (to
> >> be in jclouds/jclouds). Most APIs would be in here.
> >> 2. Those that provide provider-specific features only, and cannot be
> >> abstracted. There should only be a few here, but this means a few per
> >> provider, so overall this category might contain a large number of APIs.
> >>
> >> If we were to organize the repos using the two categories above, we can
> do
> >> something like (suggestion):
> >>
> >> 1. jclouds/jclouds (Same as now, with more APIs - including beta APIs -
> >> promoted. Regular release schedule)
> >> 2. jclouds/providers (All provider-specific APIs that will never be
> >> abstracted. Release together with 1. This also needs a better name, as
> it
> >> only contains APIs that are not abstractable).
> >> 3. jclouds/labs (very experimental only - will not be released)
> >>
> >> Note that I am strongly in favor of fewer repos for us to manage. Maybe
> we
> >> should not even have 2.
> >>
> >> What would you change in the description above?
> >> Thanks!
> >> -Zack
> >> ________________________________________
> >> From: Ignasi Barrera [ignasi.barrera@gmail.com]
> >> Sent: Monday, December 08, 2014 4:19 PM
> >> To: dev@jclouds.apache.org
> >> Subject: Re: [DISCUSS] What is the criteria for something to be in the
> >> jclouds/jclouds repo?
> >>
> >> 1. So once an api meets the criteria of having a steward, being
> >> modern, and having a proper test suite, it can immediately be
> >> promoted?
> >>
> >> I really think that those APIs need to have an abstraction, or a
> >> potential abstraction that is being developed, but something real,
> >> tangible, that ensures that it is aligned with the essence of what
> >> jclouds is.
> >>
> >> 2. Can you elaborate on this point? Did you have some criteria in mind
> >> for what “makes sense” means?
> >>
> >> There are several features more than one cloud provider offers
> >> (networking, load balancers and databases, are the obvious examples)
> >> that could have an abstraction. Not every feature in every cloud
> >> provider may make sense in others, and thus are not suitable to have
> >> an abstraction. In that case, we should be asking ourselves what value
> >> adding such an "independent" API gives to jclouds. We have provider
> >> specific APIs to *complete* the functionality "offered by the
> >> abstractions", but I really don't think is a good idea to add "entire
> >> independent apis" that won't have an abstraction because they're too
> >> specific to a single cloud provider. When I say "makes sense" I'm
> >> saying to ask ourselves if that API really makes sense in the global
> >> jclouds picture. If it makes sense, let's work on getting those
> >> features available in a portable way to other providers too. If that
> >> doesn't make sense, then that API doesn't make sense to jclouds.
> >>
> >> 3. Are you saying that if there’s an api that would never be a good
> >> fit for abstraction, that it has no chance of being promoted?
> >>
> >> Yes. I really believe we should be doing that. As I said, jclouds is
> >> not an API placeholder to every single API that qualifies itself as
> >> "cloud". jclouds is all about abstractions, and about providing users
> >> a way to write portable code. Of course we have provider specific
> >> features, but as I mentioned before, those should serve the purpose of
> >> completing the abstractions, but never be independent and standalone
> >> providers/apis. That has proven to be difficult to maintain.
> >> considerably increase the codebase complexity and make it more
> >> difficult to evolve jclouds itself.
> >>
> >> 4. Are you saying that if an abstraction doesn’t happen in some amount
> >> of time, that any apis related to that lack-of-abstraction would be
> >> removed?
> >>
> >> Well, this is open source, and "time" is relative. The complexity of
> >> abstractions can be very different: databases seems to be a simple,
> >> but networking can be really complex. What I'm saying is that we
> >> should *work* on those abstractions and *work in a direction to make
> >> those abstractions" something real. But we can't just keep adding more
> >> "satellite" APIs and forget about the rest. We should be thinking
> >> about abstractions and woking on them, and that's what matters, not
> >> time. Take the database APIs as an example. It has been commented more
> >> than once that it would be nice to have an abstraction. What actions
> >> have been taken in that direction? None. Any proposal? Nope. That's
> >> what we have to change.
> >>
> >>
> >>
> >> On 8 December 2014 at 16:01, Everett Toews <everett.toews@rackspace.com
> >
> >> wrote:
> >> > On Dec 2, 2014, at 2:31 PM, Ignasi Barrera <nacx@apache.org> wrote:
> >> >
> >> >> This said, the absence of an abstraction (take network or databases
> as
> >> >> an example) shouldn't be a blocker. Abstractions can be proposed in
> >> >> the mailing list, and even better discussed in a PR! We are all
> >> >> looking to improve jclouds to make it better for the users, and that
> >> >> means making it more portable. Let's work on adding an abstraction
> >> >> when that makes sense! But things have to be *real* and that work has
> >> >> to be done. Without it, it does not make sense to promote/keep such
> >> >> apis/providers.
> >> >
> >> > I think having everything in jclouds/jclouds could possibly work but
> we
> >> would need to be more specific about the above. Otherwise I’m concerned
> >> we’d wind up in the same place 6 months down the road.
> >> >
> >> > If this were to be the path we go down, I have some clarifying
> >> questions. These are open questions for anyone to answer.
> >> >
> >> > “the absence of an abstraction shouldn’t be a blocker”
> >> >
> >> > 1. So once an api meets the criteria of having a steward, being
> modern,
> >> and having a proper test suite, it can immediately be promoted?
> >> >
> >> >
> >> > “Let's work on adding an abstraction when that makes sense!”
> >> >
> >> > 2. Can you elaborate on this point? Did you have some criteria in mind
> >> for what “makes sense” means?
> >> >
> >> >
> >> > "it does not make sense to promote/keep such apis/providers.”
> >> >
> >> > These are two different concerns and should be broken out.
> >> >
> >> > "it does not make sense to promote such apis/providers.”
> >> >
> >> > 3. Are you saying that if there’s an api that would never be a good
> fit
> >> for abstraction, that it has no chance of being promoted?
> >> >
> >> > "it does not make sense to keep such apis/providers.”
> >> >
> >> > 4. Are you saying that if an abstraction doesn’t happen in some amount
> >> of time, that any apis related to that lack-of-abstraction would be
> removed?
> >> >
> >> >
> >> > Everett
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message