openoffice-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damjan Jovanovic <dam...@apache.org>
Subject gotoSCons: automated gbuild to SCons conversion
Date Sat, 01 Feb 2020 12:36:42 GMT
Hi

gbuild has become an unmaintainable nightmare. While there are only 37
internal dmake modules left (+ another 37 external modules), I cannot
motivate myself to convert even 1 more. At my development speed of about 2
lines of code every 8 hours, gbuild is a disaster that wastes unbelievable
amounts of time, and it's really not a development I can say I am proud of.
It has the ugliest syntax ever, it can only be debugged through
experimentation, and nobody really understands it. Also, it doesn't fully
work, eg. a lot of the newer targets I've added don't get cleaned on "make
clean", CustomTarget fails to deliver files sometimes, etc.

To that end, I've restarting playing with an old idea: to switch to the
SCons build system, using a tool to automatically convert build files from
gbuild to SCons.

I got my previous scons build (of 1 module) up and running very quickly,
and it still works. Then I continued development of my gbuild parser and
automated converter, which I've called gotoSCons, and got it to parse most
of the build instructions in Module_xxx.mk, Library_xxx.mk,
Executable_xxx.mk and Package_xxx.mk and convert them to a SCons build file
(SConscript).

gotoSCons is relatively simple, about 1200 lines of Java, and I can provide
it to interested parties, or put it in a Git branch. It already found 3
bugs in these gbuild files:
1. main/xmloff/Package_inc.mk duplicates this line:
$(eval $(call
gb_Package_add_file,xmloff_inc,inc/xmloff/XMLTextShapeImportHelper.hxx,xmloff/XMLTextShapeImportHelper.hxx))
I've committed a fix in 85bfc14eebba4af4847075b1cf1eaecfa87bcfc4
2. main/bean/Module_bean.mk adds no targets.
3. main/testgraphical/Module_testgraphical.mk does nothing.

The gotoSCons parser is very strict, it tries hard to guarantee a correct
conversion, and will immediately fail on non-deterministic sections, such
as "ifeq ($(OS),WNT)". It will also immediately fail on deliverable types
that I haven't implemented yet, such as AllLangResTarget_, StaticLibrary_
etc., and commands within targets that I haven't implemented yet such as
gb_Executable_set_targettype_gui and gb_Library_use_externals. Out of our
total of 105 gbuild modules, 53 use non-deterministic sections, and will
require some degree of manual conversion, but the converter should
eventually be able to handle the other 52 modules. Even with the limited
deliverable/command support at present, it can successfully convert 19
modules. The conversion would be straightforward even if done manually, as
both gbuild and SCons are high level build systems with similar concepts:
includes, defines, cflags, libraries to link to, and so on (dmake is the
difficult low-level tool).

I love SCons. It's been easy integrating it into our build. It just works.
It has beautiful clear syntax. It's understandable and debuggable. It makes
development fun again. It's a well established build system, with a 20 year
history, supporting Python 2 and 3, supporting almost every platform out
there including OS/2, supporting every version of MSVC including Visual
Studio 2019, supporting advanced features like ELF sonames, library version
symlinks and library name prefixes and suffixes, flex and bison targets
(which we all need), automatically generating cleaning targets for every
build target, can use checksums instead of timestamps to avoid unnecessary
rebuilds, automatic header dependency extraction, can build a module by
itself instead of the whole project, parallelizes building at a file level
(across module boundaries). It can build at least some AOO modules without
Cygwin. It is easy to add custom targets, unlike gbuild where it's almost
impossible. It is equal to or better than gbuild in just about everything,
and we don't have to maintain it going forward like we do with gbuild: its
developers develop it, and we just use it.

If someone would like to get involved, please let me know soon, so we can
decide on an API, file layout, etc. before I've begun committing any
changes (and relax, they would be committed to a separate branch until we
are convinced it works well).

I was the major force that led us into more gbuild, but now I think it's
time to leave it. It's already coming apart at the seams, and I don't see
it being developed further. It's based on ($eval), an optional extra added
quite late, as an afterthought for special cases, probably not how GNU make
was intended to be used at scale, and a feature absent in other "make"s.
The only other project in the world that uses gbuild is LO, and I recall
reading how it took them an enormous amount of work to migrate to it fully.
Let's learn from their mistake?

Regards
Damjan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message