openoffice-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From janI <j...@apache.org>
Subject Re: Build breaker and clean build
Date Fri, 12 Apr 2013 08:18:16 GMT
On 12 April 2013 09:27, Andre Fischer <awf.aoo@gmail.com> wrote:

> On 11.04.2013 18:43, Andrew Rist wrote:
>
>>
>> On 4/11/2013 12:06 AM, Andre Fischer wrote:
>>
>>> On 10.04.2013 18:57, Andrew Rist wrote:
>>>
>>>>
>>>> On 4/10/2013 1:33 AM, Herbert Dürr wrote:
>>>>
>>>>> On 2013/04/10 10:09 AM, Andre Fischer wrote:
>>>>>
>>>>>> tonight we had a build breaker in the windows build: a slot id that
is
>>>>>> used in SW had been removed in SVX.  The reference in SW had also
been
>>>>>> removed, so this change should not be a problem.
>>>>>> But the windows build is still not a full build. Therefore the old
SW
>>>>>> slot header files where used and the build broke.
>>>>>>
>>>>>> There is an easy fix for situations like this: a clean build.
>>>>>>
>>>>>
>>>>> Incremental build are known to have problems thats why I suggested [1]
>>>>> to default to a clean build. That didn't receive consensus though and
>>>>> indeed there are good reasons against it:
>>>>>
>>>>> The incremental build both tests the dependency system and it reduces
>>>>> the load when building significantly.
>>>>>
>>>>> On the already strained buildbot this means a factor of almost five
>>>>> improvement as clean build takes about 4.5h whereas an incremental build
>>>>> takes only 0.5-1.0h.
>>>>>
>>>>> Andrew even had to reschedule the snapshot build away from the weekly
>>>>> clean build because the buildbot load is a real problem.
>>>>>
>>>>> [1] http://markmail.org/message/**wmlhc5f5zaiiyu2o<http://markmail.org/message/wmlhc5f5zaiiyu2o>
>>>>> [2] http://markmail.org/message/**7q64ijlwygdqmwf3<http://markmail.org/message/7q64ijlwygdqmwf3>
>>>>>
>>>>>  Just to add here, that there are also issues with a clean build. The
>>>> clean build fails with some frequency on hung jobs and requires manual
>>>> attention.
>>>>
>>>
>>> That is one more reason to have more frequent clean builds so that we
>>> can find the cause of these problems.  They are not restricted to the build
>>> bot.   Others are affected, too.  I would have tried to fix this, but I am
>>> not able to reproduce this problem on my local machine.
>>>
>>
>> Sorry, I think I was unclear there.  Due to the complexity of our build
>> process and the interaction with the buildbot, there is a reasonably high
>> incidence of false positive failures on clean builds.  The Windows build
>> ends up with hung processes and throws an exception.  If we were to switch
>> to clean builds we could expect several false positives a week, which would
>> require manual intervention.  We have tests of clean builds daily on the
>> linux boxes, so in terms of coverage of the entire AOO buildbot setup, we
>> have full builds covered.   I see the fact that some of our builds are
>> incremental and some are clean, as a feature, not as a shortcoming.
>>
>
> OK, I understand.  I see two problems with this approach:
>
> 1. We will not find  the build problems when we continue to hide them with
> workarounds.  Fixing them would benefit others as well, not just the build
> bots.
>
> 2. The task of the build bots is not only to verify that our source code
> is buildable but also to provide download sets for developers and QA.
>
>
>>
>>>  In reality, breaking changes that require a clean build are pretty
>>>> rare.  For me, the clean build on the weekend and incremental during the
>>>> week seems to be a good compromise.
>>>>
>>>
>>> I am not sure about that.   Besides, it is sometimes a bit difficult to
>>> judge 'how incompatible' a change really is. Change a slot definition in
>>> SVX and its use in SW.  Do we need anything more? With a clean build we are
>>> on the safe side.
>>>
>> It's a trade off.  How many false positives to you get, and how much
>> manual intervention does the system require.   What I'm trying to
>> communicate is that my experience with 'this buildbot setup' suggests that
>> the current approach requires less of my time to keep healthy, and produces
>> less false positives.
>>
>
> Ah, again I understand.  It is a matter of pragmatism.  I can accept that.
>  I and I think all others on this list are thankful for your work in this
> area.  And maybe we can help to improve the current state.  Is there any
> documentation, a Wiki page maybe, to get an overview?
>
> Maybe we can rededicate one of the daily builds of one of the branches to
> debugging our windows build problems?  Add a step to the build setup to
> apply patches that provide us with more debug info?
>
If I somehow can get my hands on the windows build bot vm, I can let it run
on my own server, and thereby free up the l10n build.

rgds
Jan I.

>
>
>
>>> Besides, the clean-build-on-weekend policy would require us to hold all
>>> incompatible changes until Friday, or live with a broken build during the
>>> week.
>>> I really thing that we need a better solution.  A switch for marking a
>>> change as incompatible and that would be interpreted by the build bot would
>>> be the absolute minimum.  But even that would call for trouble.  At Sun we
>>> have been there and it did not really work so well.
>>>
>> This doesn't require waiting until the weekend, it requires a manual
>> clean run which can be kicked off easily.  (I'm happy to show you, or hit
>> up Herbert - he has access, too)
>>
>
> It would be good to have this knowledge written down somewhere for easy
> reference.  Especially if it really remains our only way to get clean
> builds.
>
>    I don't disagree with your general argument, I just see different
>> trade-offs, and I consider this type failure to have a fairly trivial
>> recovery (kicking off a manual clean build).
>>
>
> Again, I don't think that this is a trivial solution.
>
> First I have to detect that my changes are incompatible.  That is
> sometimes easy but sometimes it is not.  If it is not easy then there is
> even the possibility that the incremental build will complete successful.
>  But the result might show some very subtle errors that we may or may not
> find before a release.
> I would rather not spend any time on the analysis.
>
> Then we have to coordinate the triggering of incompatible builds. We have
> developers working around the world at every time of the day.  It might be
> better to have an incompatible build to start a a fixed time then have
> developer A trigger a clean build and developer B, who is one hour late,
> has to do that again right after that. This might flood the build servers
> with AOO builds.
>
> -Andre
>
>
>> Andrew
>>
>>>
>>>
>>>>  This may become important in the coming weeks when we have to fix some
>>>>>> bugs in the sidebar (which is about to be merged back into trunk).
>>>>>>  The
>>>>>> sidebar is implemented in several modules.  Without a clean windows
>>>>>> build we will run into build breakers very regularly.
>>>>>>
>>>>>
>>>>> It is possible to force a clean build manually.
>>>>>
>>>> I'm cleaning it up now and kicking off a build.
>>>>
>>>
>>> Thanks for taking care of this one.
>>>
>>> -Andre
>>>
>>>  Andrew
>>>>
>>>>
>>>>
>>>>> Herbert
>>>>>
>>>>>
>>>>> ------------------------------**------------------------------**
>>>>> ---------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<dev-unsubscribe@openoffice.apache.org>
>>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>
>>>>>
>>>>
>>>> ------------------------------**------------------------------**
>>>> ---------
>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<dev-unsubscribe@openoffice.apache.org>
>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>
>>>>
>>>
>>> ------------------------------**------------------------------**
>>> ---------
>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<dev-unsubscribe@openoffice.apache.org>
>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>
>>>
>>
>> ------------------------------**------------------------------**---------
>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<dev-unsubscribe@openoffice.apache.org>
>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>
>>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<dev-unsubscribe@openoffice.apache.org>
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message