openoffice-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From janI <j...@apache.org>
Subject Re: [proposal] replace build.pl with a central Makefile.
Date Fri, 18 Oct 2013 17:54:52 GMT
On 18 October 2013 16:52, Andre Fischer <awf.aoo@gmail.com> wrote:

> On 18.10.2013 15:58, janI wrote:
>
>> On 18 October 2013 15:00, Andre Fischer <awf.aoo@gmail.com> wrote:
>>
>>  On 18.10.2013 14:02, janI wrote:
>>>
>>>  sd
>>>>
>>>>
>>>> On 18 October 2013 13:36, Andre Fischer <awf.aoo@gmail.com> wrote:
>>>>
>>>>   On 18.10.2013 11:32, janI wrote:
>>>>
>>>>>   Hi.
>>>>>
>>>>>> due to the discussion in thread "Mentor a new build system", I have
>>>>>> made a
>>>>>> proposal for a central Makefile located in main.
>>>>>>
>>>>>>   Hi Jan,
>>>>>>
>>>>> it is great that you are going to improve this part of the build
>>>>> system.
>>>>>    But I think that we need more details about how the proposed build
>>>>> system
>>>>> works.  Without them I can not really evaluate the proposal.
>>>>>
>>>>>   First of all, I agree with juergens remarks that this should be
>>>>>
>>>> discussed
>>>> before implemented, hence the wiki page.
>>>>
>>>> Secondly this has nothing directly to do with the proposed build system,
>>>> its a simple replacement of build.pl in the current system.
>>>>
>>>>  Yes, that is how I understood it.  I just did not know how to call the
>>> build.pl replacement.
>>>
>>>
>>>
>>>  I know that build.pl works, but having a Makefile in main, would make
>>>> us
>>>> one step closer on being compatible with the distros. To me this job is
>>>> a
>>>> simple cleanup, not something we deadly need, but nice to have.
>>>>
>>>>
>>>>   Some remarks regarding the missing options:
>>>>
>>>>> --from <module>
>>>>>      This is one of the more important options and one that I use
>>>>> frequently
>>>>> (also in the form --all:<module>).
>>>>>      Note that if you are in <moduleA> and call 'make --from <moduleB>'
>>>>> then
>>>>> all modules are built
>>>>>      a) which <moduleA> depends on
>>>>>      b) but not those that <moduleB> depends on
>>>>>      c) Both <moduleA> and <moduleB> are built.
>>>>>
>>>>>   I have changed the documentation.
>>>>>
>>>> I use the --all:<module> myself very often, and have changed the
>>>> documentation, because it is of course supported.
>>>>
>>>> The difference is that you do the call in main, but that is a minor
>>>> detail
>>>> that can be easily corrected (have <module>/Makefile calling
>>>> main/Makefile.
>>>>
>>>> I have also changed documentation on --html due to juergens comments.
>>>>
>>>>  I am not sure that we understand --from and --since in the same way so
>>> I
>>> will try to explain what I think they do.
>>>
>>> Let's imagine that we have a simple project with modules A, B, C, D and
>>> E.
>>> where B depends on A, C on B, D on C, and E on D.
>>> A ' make all' would mean 'make E'.  The dependencies would then lead to
>>> building modules A, B, C, D, E in this order.
>>> If I am in E and call 'make --from C' then only C, D, and E should be
>>> built.  A 'make --since C' would only build D and E.
>>>
>>> If I am in D and call 'make --from B' then modules B, C, and D are built.
>>>   Call 'make --since B' to build only C and D.
>>> Note that 'make --from' accepts more than one module name (while 'make
>>> --all:<module>' does not).
>>> Note also that in the above case (stand in D, call 'make --from B')
>>> module
>>> A is not built, regardless of whether there are changes in A or not.
>>>   Whereas a simple call to make (still standing in D) would build all
>>> modules that D depends on, directly or indirectly.  Thus the options
>>> '--from' and '--since' exist to actively exclude modules from being
>>> built.
>>>
>>> The whole thing becomes a little bit more complicated with multiple
>>> options to '--from' (I never use '--since' and also don't know a valid
>>> use
>>> case so I will ignore it for now) and more complex dependencies then in
>>> the
>>> simple example above.  Let's say that if we stand in instsetoo_native and
>>> call 'make --from svx sfx2'.  Note that svx depends on sfx2.  This would
>>> build svx, sfx2 and all modules that depend (directly or indirectly) on
>>> svx
>>> OR sfx2.
>>>
>>>  got it, now I just have one problem, why would you not build the
>> dependent
>> modules, if they needed to be built, thats a scenario I dont understand.
>> With a central makefile, <module>/makefile will not be called so we do not
>> waste cpu cycles.
>>
>> With the .done files, we know when a module was last built and all modules
>> that depend it should be rebuilt which the rule
>> <module>.done : <module_depend>.done
>>
>> will ensure, so If we have A -> B -> C -> D
>>
>> I go in B, and call make, then when I go in D and make, B,C,D will be
>> made.
>>
>> If we have A -> B -> D   C -> D
>> and do the same then only D will be made.
>>
>> So --from is not really saving anything ?
>>
>
> a) In your example you first go into B then, in a second step, into D.
>  The '--from' option lets you do the same (well, not really the same, but
> see below) just from D.
>
> b) You go first to B and call make.  This makes A, if necessary, then B.
>  The making of A is exactly the thing that you want to prevent with the
> '--from' option.  Go into D and call 'make --from B'.  A is not built.
>

Actually I go to main and say "make D", that will cause B but not A to be
built.


>
> c) After the discussion with you I am not sure if we still need --from
> because the two reasons I know for its existence my not be relevant with
> the new approach.
>

Thx, I think the discussion is important. I am always afraid (still feeling
newbee) to miss something, and at the same time many other say "we are used
to". I think the right thing to do is somewhere in the middle, and we can
only reach it by discussing it.

I find discussion where e.g. you say "I need option X because" very
fruitfull, because they are based on actual need, and not just "nice to
have".


>
> c1) With the '--from' option you can tweak the dependency rules at runtime
> (a bit).  This allows you to exclude projects from being built when you
> know that that is not necessary.  But from experience I know that can lead
> to very subtle errors.  Letting the system determine what to built is
> usually more reliable.
>
you are a pro. and I am getting there, but all newcommers are not, and to
attracts them we need to make it easy and fault tolerant !

I am sure, I dont need to tell you how to tweek .done file to achieve what
you want :-)


>
> c2) With build.pl a 'build --all' still builds every module on which the
> one you are standing in depends on.  When those modules have been built
> previously, then no compilation takes place.  But calling dmake for a
> couple of directories for close to 200 modules (when you stand in
> instsetoo_native) takes a lot of time (several minutes on Windows), even
> when no file has to be compiled.  This wasteful way of doing nothing can be
> prevented with the --from option.
> However, with the new approach and its .done flag files you can determine
> which modules need to be built much faster.  You don't have to call dmake
> on directories that where already built.  Hm, but this again, does only
> work if your .done files have dependencies on all relevant source files.
> That is something that is missing at least from my script.
>

Hmmm, I had a different idea, let me try that on you. If someone changes a
source file, but does not do a make, its not intented to be public. Meaning
we do not depend on the source files, but the actual run of make. Is that a
wrong idea ?


>
>
> So, reasons for the existence of '--from' are a result of old/slow
> computers, slow files systems (still valid on Windows), missing global
> dependencies (which we now have for gbuild) and impatient developers.
>
>
>
>>
>>  While this is easy to do with eg Perl I am not sure how to handle this
>>> with just a Makefile.  The straightforward approach with handling
>>> <module>.done files does not work.  And that is one of the reasons why
I
>>> don't think that (GNU) makefiles are a good solution for any problem.
>>>  Most
>>> of us are used to program object oriented/imperative.  Makefiles require
>>> a
>>> declarative approach. Maybe the use of Perl is not such a bad idea.
>>>  Maybe
>>> it would be better to reimplement build.pl with a lot fewer options and
>>> with better readable code.
>>>
>>>  I agree that makefiles are nowhere near a good solution to many of these
>> problems, but its like windows, I dont like it, but everybody uses it.
>>
>> We could easily write a new build.pl, that also took care of the local
>> makefiles, but our build system would not be in the mainstream, and e.g.
>> the distros would not like to integrate AOO.
>>
>> I have over the last years followed research in building systems, and
>> there
>> are (sadly enough) nobody that tries a real object oriented aproach. Also
>> if you look at packages like visual studio, QT, eclipse they all use the
>> principle of makefiles with declarative approach.
>>
>> So my simple question is, do we want to approach the main road (makefiles
>> for unix, visual studio for windows/mac) or do we want to have better but
>> non standard system.
>>
>
> Good analysis.  Maybe I should answer with Faust: "Zwei Seelen wohnen,
> ach! in meiner Brust" (two souls alas! are dwelling in my breast).  The
> pragmatist says to use the make.  It is good enough for others, it is good
> enough for use.
> On the other hand, when I start a new project I usually start with the
> question of what are the best tools for the problem at hand. And make does
> not seem to be the first or the best answer.  Look at our dmake or gbuild
> system.  Both don't use make in a standard way. gmake even defines its own
> language, object oriented and imperative, on top of the GNU make macros.
>  That is, for me, an act of desperation.
>

I just wonder did you see "gütterdämmerungen" (I hope I spelled it right,
its 2 years since I lived in Vienna so its a bit rusty), I saw it on
danube, 2 nights in a row.


> I have made experiments with an alternative approach, a domain specific
> language somewhat similar to Java.  I personally like this approach because
> a) it uses the paradigm that I already use when writing C++ code. That
> means that I can apply my existing knowledge to the build process and that
> I don't have to remember all the tricks and pitfalls of makefiles.
> b) as expected it was much easier to handle file dependencies and parallel
> processing of build jobs in Java than adding object orientation and
> imperative control flow to makefiles.
>
I like your approach, and would like to help on that, BUT I think we
benefit as a community to stay on the main road.


>
> If I had the time and if I would be the one working on it then I would
> prefer an non-Makefile approach.  But maybe I am just suffering too much
> from one of the 'three great virtues of a programmer': hubris.
>

No you are like me, just not understaning why the world isnt perfect :-)


>
> You are the one who leads the build project changes, so you have to decide
> which approach to use.  I am not trying to make your life harder (than
> necessary), I am only trying to point out some of the pitfalls that I have
> encountered in the past.  And to prevent you from removing features that I
> use :-)


I do not decide things, I think about solutions, throw them at the
community, and see if the float. If they float I help implement them.

I am a doer, but also a listener. Implementing what I think is right, would
be dead wrong, but I pave the way and make us discuss/think sometimes by
provoking a bit, which I think is good.


>
>
>
>
>> rgds
>> jan I.
>>
>> Ps. its always refreshing to discuss with you, you often have an
>> alternative approach, which is not just a dream but doable.
>>
>
> Thanks. That makes two of us.
>
> Have a nice weekend,
>
same to you !

rgds
jan I.


> Andre
>
>
>>
>>  -Andre
>>>
>>>
>>>    --prepare
>>>>
>>>>>      Also one option that is important for our every day work.  Use
>>>>> case:
>>>>> You make changes in <module> and are not sure if these changes
are
>>>>> compatible/incompatible.  To be on the safe side you discard the output
>>>>> of
>>>>> all depending modules.  To save time you keep the output of all other
>>>>> modules.
>>>>>
>>>>>      Often used together with '--from' like 'make --prepare --from
>>>>> svx' to
>>>>> prepare a build after making changes in svx.
>>>>>
>>>>>   Documentation changed, funny thing is that svx does not clear
>>>>> correctly
>>>>>
>>>> on
>>>> my ubuntu build.
>>>>
>>>>
>>>>   --since <module>
>>>>
>>>>>      A variant of '--from'.  The only difference is that <module>
>>>>> itself
>>>>> is
>>>>> not built.
>>>>>
>>>>>      If your proposed approach is similar to what my script produces
>>>>> then
>>>>> it
>>>>> is not too difficult to support --from/--since.  I made some
>>>>> experiments
>>>>> in
>>>>> this direction but was to lazy to finish them.
>>>>>
>>>>>   My approach is very similar, but I failed to see how --since is
>>>>>
>>>> supported.
>>>> And question is if its real important.
>>>>
>>>>
>>>>   --job
>>>>
>>>>> --pre_job
>>>>> --post_job
>>>>>     These are sometimes handy to run a non-standard command for all
>>>>> modules.
>>>>>
>>>>>   I have added them, they are by the way a good example why we need a
>>>>>
>>>> discussion I have never used them.
>>>>
>>>> However maybe the real discussion is "do we want to replace build and
>>>> have
>>>> a main/Makefile instead?"
>>>>
>>>>
>>>>
>>>>   - I have not used the rest of the unsupported options and would not
>>>> miss
>>>>
>>>>> them.  Others may have other sets of options that are important to
>>>>> them.
>>>>>
>>>>>
>>>>> Some general remarks:
>>>>>
>>>>> - Why keep one makefile per module?  Why not put all the inter-module
>>>>> dependencies into one file (like my script does)?
>>>>>
>>>>>   Ups, I did not explain that correctly, I propose 1 Makefile
>>>>>
>>>> "main/Makefile"
>>>> with all inter-module and 1 Makefile "<module>/Makefile" that today
just
>>>> will call the old makefiles as described in prj/build.lst
>>>>
>>>> - Why not use the oportunity to move (a part of) the build environment
>>>> out
>>>>
>>>>  of the way to, say, build/ ?
>>>>>
>>>>>   You have guessed my next step.
>>>>>
>>>>
>>>>   - How are dependencies between modules handled (just the manual
>>>>
>>>>> dependencies from prj/build.lst or also the file dependencies
>>>>> introduced
>>>>> by
>>>>> gmake).
>>>>>
>>>>>   See doc. on --from. Its done with <module>.done files
>>>>>
>>>>
>>>>   - How is the output of the individual calls to dmake or GNU make
>>>>
>>>>> handled/made accessible.  Ie. if there is a build error, how can I look
>>>>> up
>>>>> the corresponding build output?
>>>>>
>>>>>   see doc. script make_log
>>>>>
>>>>   - Are the gmake makefiles included (run in the same process) or is GNU
>>>>
>>>>> make started for them it its own process?
>>>>>
>>>>>   For a start they would be called (own process), but its something
>>>>> where
>>>>>
>>>> I
>>>> have no strong opinions.
>>>>
>>>> Please (just to be sure), this proposal has nothing to do with the
>>>> students
>>>> work, its simply because I saw a positive discussion on removing
>>>> build.pl
>>>> ,
>>>> and spent a couple of hours looking at it. If there is a preference not
>>>> to
>>>> remove build.pl I will simply forget it.
>>>>
>>>> rgds
>>>> jan I.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>  Regards,
>>>>> Andre
>>>>>
>>>>>
>>>>>
>>>>>   It has been roughly tested it, thanks to a clever utility from andre.
>>>>>
>>>>>> As discussed build.pl contains a lot of options, which need to be
>>>>>> considered in a makefile.
>>>>>>
>>>>>> My suggestion is on
>>>>>> http://wiki.openoffice.org/******wiki/Build_System_Analysis:**<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**>
>>>>>> **<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**<http://wiki.openoffice.org/**wiki/Build_System_Analysis:**>
>>>>>> >
>>>>>> build.pl_versus_makefile<http:****//wiki.openoffice.org/wiki/****<http://wiki.openoffice.org/wiki/**>
>>>>>> Build_System_Analysis:build.****pl_versus_makefile<http://**
>>>>>> wiki.openoffice.org/wiki/**Build_System_Analysis:build.**
>>>>>> pl_versus_makefile<http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile>
>>>>>> >
>>>>>>
>>>>>>
>>>>>> Please feel free to edit/comment on the page. I have reduced to
>>>>>> options
>>>>>> a
>>>>>> lot, and some of them might be in use.
>>>>>>
>>>>>> thanks in advance for your comments.
>>>>>>
>>>>>>
>>>>>>   ------------------------------******--------------------------**
>>>>>> --**
>>>>>>
>>>>> --**---------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.****a**pache.org<
>>>>> http://apache.org**>
>>>>> <dev-unsubscribe@**openoffice.**apache.org<http://openoffice.apache.org>
>>>>> <dev-unsubscribe@**openoffice.apache.org<dev-unsubscribe@openoffice.apache.org>
>>>>> >
>>>>>
>>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>
>>>>>
>>>>>
>>>>>  ------------------------------****----------------------------**
>>> --**---------
>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**a**pache.org<http://apache.org>
>>> <dev-unsubscribe@**openoffice.apache.org<dev-unsubscribe@openoffice.apache.org>
>>> >
>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>
>>>
>>>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<dev-unsubscribe@openoffice.apache.org>
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message