james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefano Bagnara <apa...@bago.org>
Subject Re: [JSieve] pom's in stage
Date Tue, 09 Oct 2007 22:52:53 GMT
Robert Burrell Donkin ha scritto:
> On 10/9/07, Stefano Bagnara <apache@bago.org> wrote:
>> Robert Burrell Donkin ha scritto:
>>> On 10/8/07, Stefano Bagnara <apache@bago.org> wrote:
>>>> Robert Burrell Donkin ha scritto:
>>>>> On 10/8/07, Stefano Bagnara <apache@bago.org> wrote:
>>>>>> The alternative is still the #3 from the original post:
>>>>>>> 3) change the dependencies groupdIds/artifactIds for the "problematic"
>>>>>>> artifacts and move them to the new folders. Alter the pom to
declare
>>>>>>> this new dependencies. E.g: move javax.mail/jars/mail-1.4.jar
to
>>>>>>> org.apache.james.dep/jars/mail-1.4.jar or something similar.
Even if we
>>>>>>> don't put there the poms mvn will work creating a "standard"
pom with no
>>>>>>> dependencies and will deploy to the local repository our own
"artifacts"
>>>>>>> without clashing with official poms/artifacts from central. This
is make
>>>>>>> the offline build to work without placing bad stuff in the local
>>>>>>> repository but we will declare dependencies on artifacts not
existing on
>>>>>>> central.
>>>>>> opinions?
>>>>> #3 introduces bad meta-data into the maven repository
>>>> No, I proposed #3 because it does not put bad meta-data in the
>>>> repository. It only put there "custom" data. We should only take care to
>>>> use our own groupIds and no build will be corrupted by our poms.
>>>> Let's say we add "jsieve" or "james" to the groupId, we can be sure no
>>>> one else will have references to this artifacts.
>>> bad meta-data is bad meta-data - whether it's published in the apache
>>> repository or not
>> Maybe this depends on the definition of bad metadata. IMHO what is bad
>> is some metadata that could break something because it is not complete
>> or different from the right metadata.
> 
> unless JAMES plans to re-release the dependencies, the dependencies
> are completely fictional. tools such as maven and ivy which depend on
> good meta-data will be broken by fictional dependencies.

I don't understand the issue you describe. I just propose to use a
folder name that does not equal the groupId used by artifacts deployed
on official maven repositories.
This does not mean at all creating bad metadata.
Our pom.xml is not and will not be deployed to any repository using that
information to do anything. This is an internal tool used with the sole
purpose to create our website, javadocs and other reports.
The fact that we have a pom.xml in our source tree does not mean at all
that this pom will be and should be used by third party projects to do
anything. They should treat that pom as any other xml descriptor they
don't know.

That said I don't have the language property and the time to discuss
this issue more than this.
I have a clear opinion on what is bad metadata and what is good
metadata, and it seems to be different from your. Unfortunately other
committers are not saying anything on the topic, so I leave any choice
to you.

I just say that prior to my "stage" refactoring that introduced the
"incriminated" poms the jsieve build was referring a temporary maven
repository in my people.apache.org home:
http://people.apache.org/~bago/maven/dist-m1/
http://svn.apache.org/viewvc/james/jsieve/trunk/pom.xml?r1=441635&r2=450111
And the groupIds used in the pom had no real meaning for any other
project in the world but jsieve, because it was referring that specific
repository.
We all understood that the ~bago repository was simply an hack to let us
building the multiproject website with no big efforts, and the stage
folder introduced in every other project was meant to fix the very issue
of referencing people.apache.org as a maven repository. We agreed that
it was not good to do that for an official repository.

Please also note that james 2.3.0 and 2.3.1 have been released with a
pom.xml that was working only for us and using artifactIds that had
meanings only for us.

I really don't understand why the whole thing should be so different for
 jSieve that is a project built by ant.

Creating a pom.xml referring only artifacts available in official maven
repositories has never been in my goals. My goal was having a simple way
to build the website.

This whole discussion is complicating this too much, for me.

I prefer to have a pom.xml that works our of the box for us and have
links to groupIds that have meanings only for us. Our pom goal is to
create the website, not to provide metadata to the world. This would be
a secondary goal, and not so important to require a so long discussion,
IMHO.

> it is possible that substituting new poms may introduce obscure issues
> with maven but the risk is low whereas using fictional ids is
> gauranteed to break dependency management tools.

I don't agree.
There is no such concept of "fictional ids". We use ids we define. Maven
works like this. I don't see any single way that this would break
dependency management tools as we are not going to publish THAT pom to a
maven repository.
The only thing that will happen is that we will install locally some
artifacts under a custom groupId: I don't see any single way this could
harm another build.

Furthermore this entire discussion started from the licensing for the
poms I added, but I want to put emphasis on the fact that now it has
nothing to do with that very poms. The issues you describe have always
been there since we use maven to build our website and to build some of
our projects. I'm not sure you're understanding this.

Do you want to rename the pom.xml to mysitedescriptor.xml so that we'll
build the website using "mvn -f mysitedescriptor.xml site" ?

There is no direct link between using a pom.xml to do some processing on
a source tree and publishing a pom.xml to a maven repository. In fact
many projects do not use the pom.xml at all and they simply publish a
manually created pom.xml to the repository when they deploy.

Do you want to publish jsieve to a maven repository? Then you will have
to publish its depedencies first (james-2.3.1, mailet-2.3,
mailet-api-2.3) and then you can simply create a pom.xml with only the
dependencies informations and you can even refer the javax.mail:jar:mail
artifact with the groupId used in the official repository instead of the
one under the custom groupId we use to build the website.

>> Here I propose to create our own metadata for the very purpose to create
>> something we have the license to redistribute.
>>
>> The original issue is that we have no license to redistribute that poms,
>> right? I say let's create our own poms, and doing so we should not reuse
>> groupIds already used by third party projects that didn't grant us the
>> needed licensing.
>>
>> The concrete plan is (replace "org.apache.james.3rdparty" with what you
>> prefer):
>> rename stage/javax.mail to stage/org.apache.james.3rdparty and then
>> alter the pom from:
>>     <dependency>
>>       <groupId>javax.mail</groupId>
>>       <artifactId>mail</artifactId>
>>       <version>1.4</version>
>>     </dependency>
>> to
>>     <dependency>
>>       <groupId>org.apache.james.3rdparty</groupId>
>>       <artifactId>mail</artifactId>
>>       <version>1.4</version>
>>     </dependency>
>>
>> At this point we can decide whether to create pom.xml for this
>> groupId/artifactId. If we don't provide one maven will create a basic
>> one for us.
> 
> but this breaks tools (including maven) that rely on the meta-data being correct

First, this does not break maven when used to build or work with this
project.

Second, we don't give this pom to anyone! We chose maven to build the
website: this does not mean that people should use our pom.xml to do
anything else.
We did exactly the same thing for james server: what's the problem?

>>>> IMHO the only drawback is that we cannot publish our "jsieve" pom in the
>>>> maven repository because we have dependencies on artifacts that are not
>>>> in the repository. But this already happens for james and mailets jars,
>>>> so I don't think this is a big issue. If we want to publish jsieve in a
>>>> maven repository we'll have to fix many other problems, first (or to
>>>> avoid declaring the dependencies).
>>> i'm confused: which issues in particular?
>> That james-2.3.1.jar, mailet-2.3.jar and mailet-api-2.3.jar are not
>> present in a maven repository (and it is not just a matter of addning a
>> pom and publish them because our ant build didn't add NOTICE/LICENSE to
>> that jars as we didn't plan to redistribute them outside from our main
>> zip/tar.gz releases).
>> So we already declared dependencies on artifacts not existing in the
>> repositories (our own james server/mailet jars), #3 would simply add
>> some new artifacts that is only found in the local stage folder.
> 
> yes
> 
> but i'm still confused about what would be required than just adding
> NOTICE/LICENSE to the jars before they could be published in the maven
> repository

Creating a pom.xml for them.
And I always thought that adding a NOTICE/LICENSE to a jar means
creating a new release. I saw projects creating a new release only
because the crc of the being-VOTED artifact changed because of a
different jar tool used to rebuid the jar...

>>>>> what would maven do if we just removed all the pom's from the local repository?
>>>> If we remove the poms and leave the jars under existing
>>>> groupId/jars/artifactId-version.jar scheme then it will try to lookup
>>>> the poms online, if it finds the pom it download it, otherwise it
>>>> creates simple poms with no dependencies (they contains artifactId,
>>>> groupId and version). This maven-generated poms are the ones that could
>>>> break any other project depending on the same artifacts and building
>>>> against the same local repository
>>> by local respository do you mean the stage directory or ~/.mvn (or
>>> whatever the path is)?
>>>
>>> - robert
>> the local repository is ~/.m2/repository.
>>
>> when maven look for an artifact it looks in the local repository
>> (~/.m2/repository). If the artifact is not there, yet, then it starts
>> looking in the remote repositories. We "hacked" our poms so that we add
>> a "stage folder" that is a "remote repository" for maven but is local to
>> our project and is checked out with the sources. This is clearly an
>> hack, because this break some basic maven rule, but this was the only
>> solution to satisfy the JAMES PMC/AFS policies wrt m2 based releases.
>> If someone changed his mind about this topic please speak now, otherwise
>> I think we'll have to stick to that decision.
>> So when maven will need the org.apache.james:mailet-api:jar:2.3 artifact
>> it find it in the stage/org.apache.james/jars/mailet-api-2.3.jar file
>> and then also looks for stage/org.apache.james/poms/mailet-api-2.3.pom.
>> If it does not find the pom it will first lookup any other remote
>> repository for the pom and if not found will create a basic pom like
>> this one:
>> <project>
>>   <modelVersion>4.0.0</modelVersion>
>>   <groupId>org.apache.james</groupId>
>>   <artifactId>mailet-api</artifactId>
>>   <version>2.3</version>
>> </project>
>> (as you can see this is only a different representation of the maven
>> artifact identifier)
>>
>> I think the only "bad" pom is the one having the same groupId/artifactId
>> of an already publicly available pom but not declaring the same things
>> (declare no dependencies or different dependencies).
>> We can create as many new poms as we like under the groupIds we like and
>> they are not "bad metadata". They simply are redundant/alternative
>> metadata.
> 
> this is bad meta-data - there are releases with those ids

I don't agree.
There is a jar named mail.jar. It is distributed by sun under the CDDL
via website inside a ZIP file.
Someone decide to create a descriptor using a groupId and an artifactId.
Someone decided to use javax.mail for groupId and mail for artifactId.
Someone else used javamail  for the groupId and javamail for the artifactId.
We decided to use org.apache.james.3rdparty as groupId and mail as
artifactId.

There is no good name, as the JAR does not declare its own groupId.

In fact, following maven practice *the* BAD METADATA is the one already
existing:
https://maven-repository.dev.java.net/repository/javax.mail/poms/mail-1.4.pom
http://mirrors.ibiblio.org/pub/mirrors/maven2/javax/mail/mail/1.4/mail-1.4.pom
You can see they use the same artifactId/groupId but with different
content. THIS is bad metadata.
The WORST thing at all is that ALSO the jars are different!!
I just downloaded:
http://mirrors.ibiblio.org/pub/mirrors/maven2/javax/mail/mail/1.4/mail-1.4.jar
https://maven-repository.dev.java.net/repository/javax.mail/jars/mail-1.4.jar
And they contains different Session.class.
Maybe this worth further investigation as it may be an hacking attempt,
but at this time I just want to point out what is BAD metadata from what
is CUSTOM metadata.

The pom and the declared groupId/artifactId is not part of the release
and in fact this is the very issue we started from: we don't have a
license for that file. Most time the pom is not even created by the same
owner of the jar. Why we should consider some groupId official and some
other groupId BAD metadata??

The bad metadata is the one created by "your" solution (removing the
poms) when maven is ran with no online repository access as soon as it
will install the jars with no poms. As it will create a new pom in the
javax.mail groupId and mail artifactId, with the 1.4 version, but it
will not declare that this artifact has a dependency on activation and
other projects depending on javamail will then start failing to build on
the same machine.

>> Unfortunately we already accepted poms without license in
>> central and now we need a fix for this.
> 
> the pom licensing needs fixing centrally in any case

I agree. But we need also a solution for *now*, while we wait them to
fix every pom there (if they are fixable without altering their
groupIds/artifactIds).

>> The only thing we should care
>> about is to use a groupId that do not clash with possible 3rd party
>> groupIds (following maven convention we should use org.apache as we
>> control that domain).
> 
> maven is one tool amongst many that rely on this meta-data being
> correct. altering these ids is gaurateed to create bad meta-data.

I agree that maven is only one tool: I never thought the opposite in
this discussion.

If you use ant to build a project you don't publish the build.xml to the
maven repository, right? You instead have another file, or ant will
build one for you, that will be named jarname-version.pom and will be
published in the repository, right? Why can't we do the same if we use
"mvn" instead of "ant" ? Let me be clear: I would love to have a pom.xml
used both for building and as a descriptor for maven-repositories users,
but we don't even use it for building (as we keep using ant for jSieve),
why should we mix the 2 issues in this already complicate discussion?

But I don't agree with your definition of bad metadata for the same 2
motivations I said multiple times (1: THAT pom is not there to be
published and used by anyone else but people building our website, 2:
Using custom groupIds is not BAD metadata, but instead is suggested by
repository people every time you need to alter anything in the pom or
the referred jar)

> - robert

I decided to write all of this because I don't know what is your
knowledge of the JAMES historical usage of ant+maven and every opinion
of this complicate team, and I don't know what is your knowledge of
maven and how repositories work.

If, after this, you think you have a solution and that I'm simply not
understanding (or not agreeing with your solution) I just ask you to
check with Noel that your solution is fine with him and I will be fine
with any solution that do not complicate my work of deploying an update
to our website.

Stefano

PS: Sorry, it's been a long work day and I get stressed when I'm not
able to explain in english something that is so clear to me. And really,
I don't have objection to application of #1, #2, #3, #4 or anything
else. I have preferences and I already said them, nothing more. I simply
have concern with your reasons for choosing one instead of the other and
I hope it's more clear now. Thank you for reading the whole thing.


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message