jclouds-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zack Shoylev <zack.shoy...@RACKSPACE.COM>
Subject RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?
Date Tue, 16 Dec 2014 19:49:14 GMT
Let's work out some examples about promoting APIs and how the repos should work.

Example: I have a provider-specific API that extends the compute abstraction in a meaningful
way.
Currently I would have to:
1. Make a PR for the API in one of the labs
2. Make a PR for the abstraction in jclouds/jclouds
3. (Same as 1. in a way) Make a PR in other labs repos for other providers (as it is likely
new abstraction features would have to be supported in other providers - some of these might
be created by other people, obviously - or these features might already be supported)
4. Review and merge all of these.
5. Make PRs to promote the provider-specific APIs to jclouds/jclouds and merge those.
6. Make PRs to clean up labs repos.
Does this make sense? Am I missing something? How can we improve this?
 
________________________________________
From: Ignasi Barrera [nacx@apache.org]
Sent: Monday, December 15, 2014 3:22 PM
To: dev@jclouds.apache.org
Subject: Re: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

I can agree that giving full control over provider specific APIs can
be a good thing, but always as a secondary goal. As you say,
abstractions and working on the missing ones should be the high
priority. Adding provider specific features to complete the
abstractions is also a common sense thing; and at this concrete point
in time, keep growing or promoting the APIs that are not related to
any abstraction don't benefit the project as a whole.

I think the entire point of this discussion is to set up a baseline
that allows us to "move forward". To do that, we need to have a common
criteria and a common vision of which should be the next steps to
evolve the project, and it seems we agree that the high priority
should be to work on the abstractions and focus on them.

This means that specially the PMC and committers should work to make
this happen, and focus on this high priority tasks. Setting a rule
that we need an abstraction to promote the APIs will help us have
under control what we promote and help us focus on the tasks we have
defined as high priority. If we keep putting our major efforts in
things that are not the high priority (apis not related to an
abstraction), we won't achieve the main purpose of "moving forward".

This is not black or white, and I don't think "we will never promote
an API that doesn't have an abstraction" is an statement we should do.
However, at this point in time, we should *really enforce* any
promoted provider and API to fit in an abstraction. We have limited
resources, and we need to focus in the high priority tasks. If we
don't dedicate these few resources to the high priority tasks, we
won't succeed.


So, "will having an abstraction be a mandatory requirement forever and
ever?" I'd say No.
But that will only make sense if we work *starting today* together on
the APIs that actually fit on those abstractions, and start building
and discussing the ones that could be added. We should prioritise
that, and make that a *real* thing, leaving the unrelated
apis/providers aside. Otherwise we will be at the same point and won't
be able to move forward, regardless of what we promote.





On 15 December 2014 at 18:49, Zack Shoylev <zack.shoylev@rackspace.com> wrote:
>>> Every single API in jclouds must be aligned with the project's purpose, and
> that means *portability*.
>
> jclouds also tries to do portability "while giving you full control to use cloud-specific
features."
>
>>>  if someone contributes a patch adding support for Google Drive
>
> I was not part of that discussion, but it should be possible to add a blobstore abstraction
for this, no?
>
>>> So, once again, jclouds is *not* a cloud aggregator for any API labelled
> "cloud". It is about portable code, and every single API and provider
> should meet the project goals.
>
> jclouds allows you to write portable applications, but also allows you the flexibility
to use cloud-specific features. From a user perspective, you need to have both. You want to
be able to build infrastructure using high-level code that works with all the clouds, but
occasionally you will need the flexibility to do provider-specific things (even just because
this is how providers differentiate themselves).
>
>
> I wanted to focus on the user perspective for this discussion. In the end, users want
things to work well, with less code, and with whatever providers they choose.
> Because jclouds is modular, users don't care if jclouds "is not an API aggregator of
any API that is labelled "cloud"." - users want jclouds to work well for whatever providers
and clouds they choose.
> As such, figuring out the criteria of what should be in jclouds/jclouds is not really
something that affects users directly, but has to be done to make development easier, saner,
and to reduce current and future technical debt.
> If we had infinite resources, it would make sense, for users, to include and release
all cloud APIs and their abstractions (where such abstractions make sense) as part of jclouds.
>
> Part of this is about balance. Currently we have too many cloud-specific APIs that could
be abstracted, but are not. Clearly some of the high-priority work in jclouds is working on
abstractions. That's the case at the moment. It might be a knee-jerk reaction to limit jclouds
just to abstractable APIs because of this situation, when in the future we might also want
to give users the flexibility to also use provider-specific features or even provider-specific
APIs that might be useful when used together with higher-level, more abstract code.
>
> This is our current mission statement:
> "Apache jclouds® is an open source multi-cloud toolkit for the Java platform that gives
you the freedom to create applications that are portable across clouds while giving you full
control to use cloud-specific features."
>
> Should this be morphed into:
> "Apache jclouds® is an open source multi-cloud toolkit for the Java platform that gives
you the freedom to create applications that are portable across clouds"? Is this better for
users?
>
> Thanks, and let's keep the discussion going. Figuring things out will help both contributors
and users.
>
> ________________________________________
> From: Ignasi Barrera [nacx@apache.org]
> Sent: Wednesday, December 10, 2014 12:59 AM
> To: dev@jclouds.apache.org
> Subject: RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds
repo?
>
> I'd never do point 2 and would remove those APIs. As I've said several
> times in the thread, jclouds is not an API aggregator of any API that is
> labelled "cloud".
>
> Every single API in jclouds must be aligned with the project's purpose, and
> that means *portability*. It does not make sense to keep adding APIs that
> don't fit in the project's purpose. If we do that we would be facing the
> exact same issues we are facing now (I mentioned them in my previous email)
> regardless of the number of repos we use.
>
> To put a concrete example: jclouds is *not* the Rackspace SDK nor is the
> OpenStack java SDK. Perhaps Rackspace adopts jclouds as its SDK, but
> jclouds isn't the Rackspace SDK. And this difference is important. Perhaps
> not every single API in Rackspace makes sense in jclouds, as it would
> hardly be modelled in a portable way. In that case, we shouldn't add it, as
> it does not benefit jclouds as a whole and doesn't serve the jclouds
> purpose.
>
> Let's take another example: if someone contributes a patch adding support
> for Google Drive (which was discussed in the ML or IRC), providing an API
> to store documents, should we merge it? That API does not fit in the
> BlobStore abstraction as it does not manage blobs (raw object data) but
> documents. The answer is no, because jclouds is not about that (ir is just
> a single, isolated API to store documents out there). Without alignment
> with what jclouds targets, we shouldn't add it.
>
> So, once again, jclouds is *not* a cloud aggregator for any API labelled
> "cloud". It is about portable code, and every single API and provider
> should meet the project goals. If not, we should not add them.
> El 10/12/2014 01:54, "Zack Shoylev" <zack.shoylev@rackspace.com> escribió:
>
>> If we take this to its logical conclusion, it sounds like we can separate
>> APIs into two categories:
>>
>> 1. Those that have an abstraction or can be abstracted in the future (to
>> be in jclouds/jclouds). Most APIs would be in here.
>> 2. Those that provide provider-specific features only, and cannot be
>> abstracted. There should only be a few here, but this means a few per
>> provider, so overall this category might contain a large number of APIs.
>>
>> If we were to organize the repos using the two categories above, we can do
>> something like (suggestion):
>>
>> 1. jclouds/jclouds (Same as now, with more APIs - including beta APIs -
>> promoted. Regular release schedule)
>> 2. jclouds/providers (All provider-specific APIs that will never be
>> abstracted. Release together with 1. This also needs a better name, as it
>> only contains APIs that are not abstractable).
>> 3. jclouds/labs (very experimental only - will not be released)
>>
>> Note that I am strongly in favor of fewer repos for us to manage. Maybe we
>> should not even have 2.
>>
>> What would you change in the description above?
>> Thanks!
>> -Zack
>> ________________________________________
>> From: Ignasi Barrera [ignasi.barrera@gmail.com]
>> Sent: Monday, December 08, 2014 4:19 PM
>> To: dev@jclouds.apache.org
>> Subject: Re: [DISCUSS] What is the criteria for something to be in the
>> jclouds/jclouds repo?
>>
>> 1. So once an api meets the criteria of having a steward, being
>> modern, and having a proper test suite, it can immediately be
>> promoted?
>>
>> I really think that those APIs need to have an abstraction, or a
>> potential abstraction that is being developed, but something real,
>> tangible, that ensures that it is aligned with the essence of what
>> jclouds is.
>>
>> 2. Can you elaborate on this point? Did you have some criteria in mind
>> for what “makes sense” means?
>>
>> There are several features more than one cloud provider offers
>> (networking, load balancers and databases, are the obvious examples)
>> that could have an abstraction. Not every feature in every cloud
>> provider may make sense in others, and thus are not suitable to have
>> an abstraction. In that case, we should be asking ourselves what value
>> adding such an "independent" API gives to jclouds. We have provider
>> specific APIs to *complete* the functionality "offered by the
>> abstractions", but I really don't think is a good idea to add "entire
>> independent apis" that won't have an abstraction because they're too
>> specific to a single cloud provider. When I say "makes sense" I'm
>> saying to ask ourselves if that API really makes sense in the global
>> jclouds picture. If it makes sense, let's work on getting those
>> features available in a portable way to other providers too. If that
>> doesn't make sense, then that API doesn't make sense to jclouds.
>>
>> 3. Are you saying that if there’s an api that would never be a good
>> fit for abstraction, that it has no chance of being promoted?
>>
>> Yes. I really believe we should be doing that. As I said, jclouds is
>> not an API placeholder to every single API that qualifies itself as
>> "cloud". jclouds is all about abstractions, and about providing users
>> a way to write portable code. Of course we have provider specific
>> features, but as I mentioned before, those should serve the purpose of
>> completing the abstractions, but never be independent and standalone
>> providers/apis. That has proven to be difficult to maintain.
>> considerably increase the codebase complexity and make it more
>> difficult to evolve jclouds itself.
>>
>> 4. Are you saying that if an abstraction doesn’t happen in some amount
>> of time, that any apis related to that lack-of-abstraction would be
>> removed?
>>
>> Well, this is open source, and "time" is relative. The complexity of
>> abstractions can be very different: databases seems to be a simple,
>> but networking can be really complex. What I'm saying is that we
>> should *work* on those abstractions and *work in a direction to make
>> those abstractions" something real. But we can't just keep adding more
>> "satellite" APIs and forget about the rest. We should be thinking
>> about abstractions and woking on them, and that's what matters, not
>> time. Take the database APIs as an example. It has been commented more
>> than once that it would be nice to have an abstraction. What actions
>> have been taken in that direction? None. Any proposal? Nope. That's
>> what we have to change.
>>
>>
>>
>> On 8 December 2014 at 16:01, Everett Toews <everett.toews@rackspace.com>
>> wrote:
>> > On Dec 2, 2014, at 2:31 PM, Ignasi Barrera <nacx@apache.org> wrote:
>> >
>> >> This said, the absence of an abstraction (take network or databases as
>> >> an example) shouldn't be a blocker. Abstractions can be proposed in
>> >> the mailing list, and even better discussed in a PR! We are all
>> >> looking to improve jclouds to make it better for the users, and that
>> >> means making it more portable. Let's work on adding an abstraction
>> >> when that makes sense! But things have to be *real* and that work has
>> >> to be done. Without it, it does not make sense to promote/keep such
>> >> apis/providers.
>> >
>> > I think having everything in jclouds/jclouds could possibly work but we
>> would need to be more specific about the above. Otherwise I’m concerned
>> we’d wind up in the same place 6 months down the road.
>> >
>> > If this were to be the path we go down, I have some clarifying
>> questions. These are open questions for anyone to answer.
>> >
>> > “the absence of an abstraction shouldn’t be a blocker”
>> >
>> > 1. So once an api meets the criteria of having a steward, being modern,
>> and having a proper test suite, it can immediately be promoted?
>> >
>> >
>> > “Let's work on adding an abstraction when that makes sense!”
>> >
>> > 2. Can you elaborate on this point? Did you have some criteria in mind
>> for what “makes sense” means?
>> >
>> >
>> > "it does not make sense to promote/keep such apis/providers.”
>> >
>> > These are two different concerns and should be broken out.
>> >
>> > "it does not make sense to promote such apis/providers.”
>> >
>> > 3. Are you saying that if there’s an api that would never be a good fit
>> for abstraction, that it has no chance of being promoted?
>> >
>> > "it does not make sense to keep such apis/providers.”
>> >
>> > 4. Are you saying that if an abstraction doesn’t happen in some amount
>> of time, that any apis related to that lack-of-abstraction would be removed?
>> >
>> >
>> > Everett
>>

Mime
View raw message