buildr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Assaf Arkin" <>
Subject Re: request for enhancement: compile, package and artifacts support for C++
Date Mon, 28 Jul 2008 20:25:46 GMT
On Mon, Jul 28, 2008 at 2:42 AM, Ittay Dror <> wrote:
> Hi,
> I'm working on adding C++ support to buildr. I already have a prototype that
> builds libraries and executables in Linux. I'd like to share some of the
> difficulties I had and request changes to buildr to accommodate C++ more
> easily. (Right now, I've created parallel route to that of building
> Java-like code)
> compile
> ========
> overview
> --------------------
> the compile method in project returns a CompileTask that is generic and uses
> a Compiler instance to do the actual compilation. In C++, compilation is
> also dependency based (.o => .cpp, sometimes precompiling headers). Also,
> the same code can produce several results (static and shared libraries, oj
> files with debug, profiling, preprocessor defines turned on and off). [1]
> there is the 'build' task, which is used as a stub to attach dependencies
> to.
> suggestion
> ---------------------
> * there should be an array of compile tasks (as in packages)
> * #compile should delegate the call to a factory method which returns a task
> (again, as in packages)

Yes.  And I know a few people just waiting for the change to compile
multiple things in the same project, so here's another reason for
adding this feature.

But I have to warn you, it's not as simple as it looks, I took a stab
at it before and deciding to downscale support to one compiler per
project.  It's worth doing because a lot of languages would benefit
from it, but that's also what makes it tricky.  I think it would be
easier to get C support working without it first, and separately work
on this feature and then improve C support using it.

> * generic pre-requisites (like 'resources') should either be tacked on
> 'build' (relying on order of prerequisites), or the compile task can be
> defined to be a composite (that is, from the outside it is a single task,
> but it can use other tasks to accomplish its job).

compile already is: resources is a prerequisite for compile, some
other tasks (e.g. byte code enhancing) are tacked on to compile by
enhancing it.

> package & artifacts
> =========
> overview
> ---------------
> buildr has a cool concept that all dependencies (in 'compile.with') are
> converted to tasks that are then simple rake dependencies. However, the
> conversion is not generic enough. to compile C++ code against a dependency
> one needs 2 paths: a folder containing headers and another containing
> libraries. To put this in a repository, these need to be packaged into one
> file. To use after pulling from the repository, one needs to unpack. So a
> task representing a repository artifact is in fact an unzip task, that
> depends on the 'Artifact' task to pull the package from a remote repository.

Let's take Java for example, let's say we have a task that depends on
the contents of another WAR.  Specifically the classes (in
WEB-INF/classes) and libraries (WEB-INF/lib).  A generic unzipping
artifact won't help much, you'll get the root path which is useless.
You need the classes path for one, and each file in the lib (pointing
to the directory itself does nothing interesting).  It won't work with
EAR either, when you unzip those, you end up with a WAR which you need
to unzip again.

But this hypothetical task that uses WAR could be smarter.  It
understands the semantics of the packages it uses, and all these
packages follow a common convention, so it only needs to unpack the
portions of the WAR it cares about, it knows how to construct the
relevant paths, one to class and one to every JAR inside the lib

I think the same analogy applies to C packages.  If by convention you
always use include and lib, you can unpack only the portion of the
package you need, find the relevant paths and use them appropriately.

> furthermore, when building against another project, there is no need to pack
> and unpack in the repository. one can simply use the artifacts produced in
> the 'build' phase of the other project.

Yes.  Right now it points to the package, which gets invoked and so
packs everything, whether you need the packing or not.  You don't,
however, have to unpack it, if you know the packaging type you can be
smarter and go directly to the source.

> finally, in C++ in many cases you rely on a system library.
> in all cases the resulting dependency is two-fold: on a include dir paths
> and on a library paths. note that these do not necessarily reside under a
> shared folder. for example, a dependency on another project may depend on
> two include folders: one just a folder in the source tree, the other of
> generated files in the target directory
> suggestion
> -------------------
> While usage of Buildr.artifacts is only as a utility method, so one can
> easily write his own implementation and use that, I think it will be nice to
> be able to get some reuse.
> * when given a project, use it as is (not 'spec.packages'), or allow it to
> return its artifacts ('spec.artifacts').

Yes.  Except we're missing that whole dependency later (that's
something 1.4 will add).  Ideally the project would have dependency
lists it can populates (at least compile and runtime), and other
projects can get these dependency lists and pick what they want.  So
the compile dependency list would be the place to put headers and
libraries, without having to package them.  We don't have that right

> * if a symbol, recursively call on the spec from the namespace
> * if a struct, recursively call
> * otherwise, classify the artifact and call a factory method to create it.
> classification can be by packaging (e.g. jar). but actually, i don't have a
> very good idea here. note that for c++, there need to be a way of defining
> an artifact to look in the system for include files and libraries  (maybe
> something like 'openssl:system'? - version and group ids are meaningless).
>  * the factory method can create different artifacts. for c++ there would be
> RepositoryArtifact (downloads and unpacks), ProjectArtifact (short circuit
> to the project's target and source directories) and SystemArtifact.
> I think that the use of artifact namespaces can help here as it allows to
> create a more verbose syntax for declaring artifacts, while still allowing
> the user to create shorter names for them. (as an example in C++ it will
> allow me to add to the artifact the list of flags to use when
> compiling/linking with it, assuming they're not inherent to the artifact,
> e.g. turn debug on). The factory method receives the artifact definition
> (which can actually be defined by each plugin) and decides what to do with
> it.

1.4 will have a better dependency mechanism, and one thing I looked at
is associating meta-data with each dependency.  So perhaps that would
address things like compiling/linking flags.

> I hope all this makes sense, and I'm looking forward to comments. I intend
> to share the code once I'm finished.

Unfortunately, the last time I wrote C code was over tens years ago,
so my rustiness is showing.  I'm sure I missed some points because of


> Thank you,
> Ittay
> Notes:
> [1] I don't consider linking a library as packaging. First, the obj files
> are not used by themselves as in other languages. Second, packaging is
> required to manage dependencies, because in order for project P to be built
> against dependency D, D needs to contain both headers and libraries - this
> is the package.
> --
> --
> Ittay Dror <>

View raw message