asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Westmann <>
Subject Re: Merging of AsterixDB and Hyracks repositories
Date Sun, 10 May 2015 23:46:48 GMT

> On May 10, 2015, at 10:52 AM, Ian Maxon <> wrote:
> The only reason this is an issue (IMO) is because of how we develop being a
> poor fit with Maven's versioning system. If maven let us have finer grained
> snapshot versions that arent necessarily linear, that would solve it. To my
> knowledge though, that isn't possible. VXQuery and Pregelix don't have this
> problem; they just rely on stable hyracks releases.

I think that’s not entirely correct. There are changes in VXQuery that required Hyracks
AFAIK the way that Preston did this for some of his Algebricks changes was by
1) developing the feature in VXQuery and Hyracks,
2) submitting the Hyracks part,
3) waiting/pushing for a Hyracks release,
4) changing the Hyracks dependency to the new stable version, and
5) pushing to VXQuery master.
And I think that - while a bit inconvenient - this wasn’t a really big problem, as one can
work well with a feature branch in Hyracks and another feature branch in the dependent. (I
hope Preston will correct me if I misrepresented this.)

And I guess that probably happened similarly for Pregelix in the past ...


> That would be harder in
> AsterixDB, or at least that's what the frequent use of the topic field to
> link builds leads me to believe.
> -Ian
> On May 8, 2015 18:06, "Chris Hillery" <> wrote:
>> Apache-specific issues aside, I must admit it would be a bit disappointing
>> to have to join Hyracks and Asterix into a single project base. It would be
>> convenient, but convenience breeds apathy. We solve the cross-product
>> releasing issues for Asterix, which makes us less likely to buckle down and
>> solve them for other Hyracks consumers like VXQuery and hopefully others in
>> the future.
>> Ceej
>> aka Chris Hillery
>> On Fri, May 8, 2015 at 2:47 PM, Ian Maxon <> wrote:
>>> I see your point, that is true. In this case a release of just Hyracks
>>> would also be visible in the AsterixDB commit log and vice-versa. I'm not
>>> certain what this means (or if it matters) on the Apache front. Is
>> having a
>>> sub-project, that keeps its own version an unprecedented thing?
>>> Agreed about not rushing through with this though. I think we should
>>> certainly wait until after the upcoming 0.8.7 release to actually commit
>> to
>>> any of this.
>>> -Ian
>>> On Fri, May 8, 2015 at 2:29 PM, Till Westmann <> wrote:
>>>> I'm not sure about that. An Apache release will be a source code
>> release
>>>> and not a binary release. We can have binary "convenience artifacts",
>> but
>>>> the official release is the source release.
>>>> Usually source releases are tagged in revision control such that the
>>>> content of the source archive agrees with the tag. Now if we have all
>> the
>>>> code in a same repository, I am not sure how that will work. I'm not
>>> saying
>>>> that it doesn't work, but I'm not sure how to do that.
>>>> I think that it would be good to make a full Apache release of both
>>>> projects first, such that we have a clear understanding how to do that
>>>> before we change the project layout.
>>>> Cheers,
>>>> Till
>>>> On 8 May 2015, at 13:58, Ian Maxon wrote:
>>>> Releasing would be the same, probably simpler actually. I suppose I
>>>>> haven't
>>>>> tried it so I can't be totally certain, but performing 'mvn release'
>> in
>>> a
>>>>> module directly doesn't do anything different than when it is run
>> from a
>>>>> higher-up pom as a submodule. Nothing would change if a user is
>>> dependent
>>>>> on a stable version of Hyracks, because they only ever see binary
>>>>> artifacts
>>>>> from Maven. 'hyracks' will still be called 'hyracks' and have the same
>>>>> coordinates in Maven.
>>>>> - Ian
>>>>> On Fri, May 8, 2015 at 1:47 PM, Till Westmann <>
>> wrote:
>>>>> Hmm, and what do we do about the other dependents of Hyracks (e.g.
>>>>>> VXQuery)?
>>>>>> We had separate releases of Hyracks for those in the past.
>>>>>> How would releases (branching, tagging ...) work in that case?
>>>>>> Cheers,
>>>>>> Till
>>>>>> On 8 May 2015, at 13:17, Ian Maxon wrote:
>>>>>> Hi all,
>>>>>>> An idea was brought up today in the meeting (I believe by Yingyi)
>> for
>>>>>>> solving the issues we have right now with maven project
>>>>>> interdependencies.
>>>>>>> The idea is to just merge AsterixDB and Hyracks into one git
>>> repository,
>>>>>>> and to have them as separate maven projects with a top level
>>> joining
>>>>>>> them. We actually have part of this implemented already (in the
>>>>>> folder
>>>>>>> a pom.xml exists for this). Doing this change would eliminate
>>>>>> necessity
>>>>>>> of the topic field hack in Gerrit, as well as ensure changes
>>> Hyracks
>>>>>>> don't break AsterixDB.
>>>>>>> I went ahead and made a branch that has this change implemented,
>>> please
>>>>>>> take a look at
>>>>>>> to get an idea of what's proposed. I merged the Hyracks repository
>>> into
>>>>>>> a
>>>>>>> subtree of the asterix repository- so all of the commit history
>>>>>>> merged
>>>>>>> properly. I think we would want to not commit this change through
>>>>>>> Gerrit,
>>>>>>> because if we did all of the Hyracks commit history would not
>>>>>> included,
>>>>>>> which would be unfortunate.
>>>>>>> - Ian

View raw message