qpid-proton mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rafael Schloming <...@alum.mit.edu>
Subject improving cross language maintainability
Date Thu, 19 Dec 2013 13:49:59 GMT
Hi Everyone,

I've been struggling with some of the cross language maintenance aspects of
proton for a while, and I believe we need to take some steps to improve the
situation. I'm one of a tiny number of people (two possibly) who regularly
commit changes to both the Java and C codebase and attempt to keep the two
in sync and at feature parity. Because of this, not everyone is necessarily
aware of the process, but to summarize the issue, currently there are far
too many moving parts and layers of indirection involved. This is not only
a significant drag on my personal productivity, but perhaps more
importantly it is a significant barrier to growing the number of proton
contributors in general as all those moving parts and layers of indirection
need to be understood before being able to make complete contributions.

To get an idea of what's going on here I think it helps to look at what's
involved in a simple change. Recently I noticed an edge case around status
updates in the messenger interface. The fix involved adding another value
to messenger's status enum and making a trivial logic change to make use of
that value under the appropriate circumstances. You can look at the full
commit here[1] if you like, but it breaks down in the following way:

Changes to C implementation (.h and .c file): 8 LOC
Changes to the python test shim for C: 3 LOC
Changes to the Java API: 2 LOC
Changes to the pure Java implementation: 6 LOC
Changes to the JNI binding: 4 LOC
Changes to the python test shim for Java: 3 LOC
Changes to the python test suite: 2 LOC

Now obviously from a personal productivity perspective it is at a minimum
annoying to have to touch so many different parts in order to make even a
trivial change, but setting that aside for the moment, what is really
sobering about this is that each one of the above parts involve a non
trivial learning curve on their own, and while it's true that only a few
lines of code are needed in each area, it is necessary to understand each
piece before being able to write the correct few lines of code. This
presents a pretty daunting hurdle for bringing new contributors to the
codebase up to the level needed to make even a trivial change like the one

The JNI binding and the python test shim for Java both exist to serve
similar purposes, namely to facilitate running a common test suite against
both implementations in order to ensure common behaviour. The python test
shims allow the python test suite to run against both proton-c and proton-j
(via jython), and the JNI binding allows pure  Java tests to be run against
proton-c as well as proton-j.

Currently by my count there are about 19 such tests in Java. By comparison
there are about 266 tests in the python test suite. Also, judging by
commits, the python test suite is growing/actively maintained, and the java
system tests (the subset of java tests that are run against both
implementations) are neither. On top of this the JNI binding itself has
suffered significantly from bit rot. As far as I know it is not made use of
outside of our own test infrastructure, and currently about 50% of the
tests run against it are skipped because it is only minimally maintained
when necessary to get the tests to pass.

Because of all this I'm proposing that we remove the JNI binding and roll
back the Java API/impl split that was (hastily) done to facilitate the test
infrastructure. This should significantly simplify the development process
relative to what it is now, and while there is still more learning curve
than desired with the python shims, I believe this will put us in a
position to improve the shims, remove duplication, and minimize the
overhead associated with them, ultimately allowing us to make the codebase
more transparently communicate its design and  hopefully lessen the
learning curve for new contributions.

I would love to hear thoughts and/or alternative ideas on how to improve
things. I would like to start addressing this early in the 0.7 development

[1] http://svn.apache.org/r1551950

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message