tinkerpop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Mallette <spmalle...@gmail.com>
Subject Re: [DISCUSS] ASF Board Draft Report - April 2020
Date Mon, 30 Mar 2020 18:06:11 GMT
Josh Perryman suggested some more color be added to the
3.4.5--hotfix-->3.4.6. I thought that was a good idea and ended up
re-writing most of the report. Please re-read:

---------------------------------------------------------------------------

## Description:
Apache TinkerPop is a graph computing framework for both graph databases
(OLTP) and graph analytic systems (OLAP).

## Activity:
TinkerPop released versions 3.3.10 and 3.4.5. Both were releases containing
important bug fixes and minor improvements. Within a week of those releases
being available, we received a report of a bug on our user mailing list[1]
in 3.4.5. We deemed the bug especially bad that an immediate patch release
was required and quickly turned around 3.4.6. The bug did not affect 3.3.10.

We've not had to release this sort of "hotfix" since 2011 which goes back to
a time prior to TinkerPop being an Apache project (2015) and covers over
forty releases here at a Apache plus many more prior to that. We attribute
this long run to of good releases to both our code review process and to our
test suites and procedures.

Our code review process not only gets additional contributors to provide
their feedback for a particular change, but also allows for a "pause" of up
to seven days before merging to a release branch. This pause often gives the
original contributor time to think through their body of work further and
can lead to additional testing, documentation or other quality enhancements.
The process is generally designed to slow major changes from reaching
release branches, while also leaving flexibility for smaller, lesser risk
modifications to move through that process more quickly.

As for our test suites and procedures, like most projects we have unit and
integration tests, however we also have the added complexity of testing
across five different programming languages with Java, Groovy, C#, Python,
and Javascript. Through various methods we've managed to unify tests of all
of these programming language environments under Maven, so that there is a
single way to execute all tests every time we build. In this way, for
example, we can immediately tell if a change to Java components have somehow
affected the viability of Python components without having to run some
separate test suite in a Python environment. When this approach is combined
with our extensive test infrastructure consisting of over 30,000 unit and
integration tests, we find ourselves having a reasonably high degree of
confidence that a particular change does not introduce regressions.

Of course, as the case of the failed 3.4.5 release demonstrates, no review
process, high test count or testing procedure will be perfect. Here we can
attribute the failed release to an assumption that an obvious feature was
being tested in all reasonably possible ways, but when underlying behavior
for an indirectly releated capability shifted for 3.4.5 we failed to realize
the subtle side-effect that increased the testing dimensions. This
particular
case shows that there are still obscure gaps in our test suite, but we’ve
addressed this gap and hope for another nine years without requirement of a
similar hotfix.

Aside from the excitement of the hotfix release, there are other points
worth noting. During this reporting period, the community added a Slack
channel to help enable a bit more real-time chatter about graphs, Gremlin
and TinkerPop.

We saw the growth of the wider TinkerPop ecosystem with the following
release announcements:

* jupyter-gremlin-kernel[2] - a Jupyter kernel for Gremlin
* gremlify[3] - a Gremlin workspace for queries and visualization
* gremlint[4] - a Gremlin code formatter
* Gremlin++[5] - a C++ interpreter for the Gremlin language
* BitGraph[6] - a C++ Backend for the Gremlin with GPU Acceleration

All of these are interesting new additions to the ecosystem. The Jupyter
integration is especially useful in that it puts Gremlin into the hands of
Jupyter users, which is a fairly popular ecosystem in and of itself. We
also like the addition of gremlify which will hopefully make learning about
Gremlin easier for folks who want to try it out without having to download
anything to their local system. The gremlint project is an incredibly useful
tool for cleaning and formatting Gremlin. We would hope that in the future
that it might have some more official standing within our Apache project.
Finally, Gremlin++ and the related BitGraph represent the first open source
implementation of Gremlin Virtual Machine outside of our project. While this
project is still under development, it represents an important part of
TinkerPop's vision where graph queries written in Gremlin in any programming
language can then be ubiquitously executed in any graph database/processor
developed in any programming language.

There are typically a number of talks and papers about TinkerPop, Gremlin
and related projects that occur during a reporting period, but it seemed
that there were less that came to light than usual. Perhaps some of this was
due to COVID-19 which cancelled a number of meetings and events. One such
case we are aware of was the cancellation of the DC Apache Roadshow where
our PMC Chair, Stephen Mallette, was scheduled to talk about Gremlin.

Committer, Josh Shinavier, did however speak at Global Graph Summit to give
his "TinkerPop 2020" presentation[7] and while we normally only list the
works of committer and PMC members for purpose of this report, we feel that
the paper of the aforementioned Gremlin++ and BitGraph distinguished itself
to be worthy of report. This paper was titled "Gremlin++ & BitGraph:
Implementing The Gremlin Traversal Language and a GPU-Accelerated Graph
Computing Framework"[8] and was authored by Alexander Barghi.

As an additional note along these lines, the paper mentioned in our last
report "Let’s build Bridges, not Walls – SPARQL Querying of TinkerPop Graph
Databases with sparql-gremlin"[9], primarily authored by committer, Harsh
Thakkar, received the Best Paper Award at ICSC 2020 the 14th IEEE
International Conference.

## Issues:
There are no issues requiring board attention at this time.

## Releases:
- 3.3.10 (February 3, 2020)
- 3.4.6 (February 20, 2020)

## PMC/Committer:
- Last PMC addition was Jorge Bay-Gondra - October 2018
- Last committer addition was Divij Vaidya - November 2019

## Links
[1] https://groups.google.com/d/msg/gremlin-users/wB0a9uP98Zw/zoZrNk8GFwAJ
[2] https://github.com/gdbassett/jupyter-gremlin-kernel
[3] https://gremlify.com/
[4] https://gremlint.com/
[5] https://github.com/bgamer50/Gremlin-
[6] https://github.com/bgamer50/BitGraph
[7] https://www.slideshare.net/joshsh/tinkerpop-2020
[8] https://drum.lib.umd.edu/handle/1903/21916
[9] https://twitter.com/Harsh9t/status/1225221027711475713


On Fri, Mar 27, 2020 at 4:53 PM Stephen Mallette <spmallette@gmail.com>
wrote:

> Here is the attached draft of our board report for this month. Please let
> me know if you'd like to see something changed or added. Did I miss any
> talks or papers (we seem short there this time around)?
>
> ---------------------------------------------------------------------------
>
> ## Description:
> Apache TinkerPop is a graph computing framework for both graph databases
> (OLTP) and graph analytic systems (OLAP).
>
> ## Activity:
> TinkerPop released versions 3.3.10 and 3.4.5. Both were release contained
> important bug fixes and minor improvements. Within a week of those releases
> being available, we received a report of a bug on our user mailing list[1]
> in 3.4.5. We deemed the bug especially bad that an immediate patch release
> was required and quickly turned around 3.4.6. The bug did not affect
> 3.3.10.
> We've not had to release this sort of "hotfix" since 2011 which goes back
> to our pre-Apache days.
>
> During this reporting period, the community added a Slack channel to help
> enable a bit more real-time chatter about graphs, Gremlin and TinkerPop.
>
> We saw the growth of the wider TinkerPop ecosystem with the following
> release announcements:
>
> * jupyter-gremlin-kernel[2] - a Jupyter kernel for Gremlin
> * gremlify[3] - a Gremlin workspace for queries and visualization
> * gremlint[4] - a Gremlin code formatter
> * Gremlin++[5] - a C++ interpreter for the Gremlin language
> * BitGraph[6] - a C++ Backend for the Gremlin with GPU Acceleration
>
> All of these are interesting new additions to the ecosystem. The Jupyter
> integration is especially useful in that it puts Gremlin into the hands of
> Jupyter users which is a fairly popular ecosystem in and of itself. We
> also like the addition of gremlify which will hopefully make learning
> about
> Gremlin easier for folks who want to try it out. gremlint is an incredibly
> useful tool for cleaning and formatting Gremlin. We would hope that in the
> future that it might have some more official standing within our Apache
> project. Finally, Gremlin++ and the related BitGraph represent the first
> open source implementation of Gremlin Virtual Machine outside of our
> project. While this project is still under development, it represents an
> important part of TinkerPop's vision where graph queries written in Gremlin
> in any programming language can then be ubiquitously executed in any graph
> database/processor developed in any programming language.
>
> There were a number of talks/papers about TinkerPop, Gremlin and related
> projects during this reporting period. Here were some by TinkerPop
> committers/PMC members, though we also include the paper for Gremlin++ and
> BitGraph:
>
> * Gremlin++ & BitGraph: Implementing The Gremlin Traversal Language and a
>     GPU-Accelerated Graph Computing Framework[7] - Alexander Barghi
> * TinkerPop 2020 - Global Graph Summit[8] - Josh Shinavier
>
> As an additional note along these lines, the paper mentioned in our last
> report "Let’s build Bridges, not Walls – SPARQL Querying of TinkerPop Graph
> Databases with sparql-gremlin"[9], primarily authored by committer Harsh
> Thakkar, received the Best Paper Award at ICSC 2020 the 14th IEEE
> International Conference.
>
> Stephen Mallette also a talk scheduled for the DC Apache Roadshow, but that
> was cancelled given the current situation with coronavirus.
>
> ## Issues:
> There are no issues requiring board attention at this time.
>
> ## Releases:
> - 3.3.10 (February 3, 2020)
> - 3.4.6 (February 20, 2020)
>
> ## PMC/Committer:
> - Last PMC addition was Jorge Bay-Gondra - October 2018
> - Last committer addition was Divij Vaidya - November 2019
>
> ## Links
> [1] https://groups.google.com/d/msg/gremlin-users/wB0a9uP98Zw/zoZrNk8GFwAJ
> [2] https://github.com/gdbassett/jupyter-gremlin-kernel
> [3] https://gremlify.com/
> [4] https://gremlint.com/
> [5] https://github.com/bgamer50/Gremlin-
> [6] https://github.com/bgamer50/BitGraph
> [7] https://drum.lib.umd.edu/handle/1903/21916
> [8] https://www.slideshare.net/joshsh/tinkerpop-2020
> [9] https://twitter.com/Harsh9t/status/1225221027711475713
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message