phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Istvan Toth <st...@apache.org>
Subject [DISCUSS] phoenix-connectors release criteria
Date Tue, 08 Dec 2020 14:59:38 GMT
Hi!

We are getting to the point where releasing the next Phoenix versions is
within sight.

The next steps will be releasing queryserver and connectors projects.
Queryserver is mostly in good shape, but the situation with Connectors is
not so clear-cut.

We (mostly Richard Antal ) have made a huge step in collecting all of the
connectors patches for 5.x, and consolidating the Phoenix 4 and 5
connectors the connectors repo.

However, not all connectors received equal attention. While all of them
compile, and run their test suite successfully, this doesn't necessarily
mean that they are useful in the real world.

Hive:
* Supports latest Hive versions (with caveats)
* Custom shaded connector JAR
* local e2e test successful for Hive 3 / Phoenix 5

Spark:
* Supports Spark 2.4 (not tested with 3.x)
* Custom shaded connector JAR (waiting for review)
* local e2e test successful for Phoenix 5

Pig:
* The last pig release (0.17) was in 2017 - abandoned ?
* connectors on version 0.13 from 2014
* (but builds and runs test suite with 0.17 with trivial changes)
* has a basic shaded jar
* I have made no effort to do e2e testing

Flume:
* Project seems to be alive, if low activity, last version is 1.9.0 from
2019
* connectors on version 1.4.0 from 2014
* (but builds and runs test suite with 1.9.0)
* has no shaded JAR (was in phoenix-client pre-split)
* I have made no effort to do e2e testing

Kafka:
* Last version is 2.6.0
* connectors on version 0.9.0.0 from 2015
* (couldn't build connector with the latest release)
* depends on and reuses flume connector code
* has a shaded jar
* I have made no effort to do e2e testing

So basically we have the Spark and Hive connectors, that me and my
colleagues actively maintain as part of $dayjob, and expect to continue to
do so in the future.

The other three, Flume, Pig, and Kafka haven't seen any development (apart
from build system updates and such) in 5-6 years, evidenced by the ancient
releases that we build with.

At $dayjob we are considering taking up the kafka connector at some point
in the future, but it will definitely not happen in the 4.16/5.1
timeframe, if ever.

As having a connectors release with some known useful and up-to-date
connectors is better than not having any connectors release at all, I
suggest that we go ahead and release phoenix-connectors 6.0.0 after 5.1 /
4.16 in the present state, but mark the Flume, Pig and Hive connectors as
unmaintained in the README / Release notes.

This leaves the door open for maintainers to step up and update/maintain
those connectors, while it lets us make regular releases for the maintained
connectors.

Please share your opinion on this plan.

regards
Istvan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message