nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tony Kurc <trk...@gmail.com>
Subject Re: Will you accept contributions in Scala?
Date Sat, 10 Feb 2018 23:43:31 GMT
It is like Matt read my mind.

On Sat, Feb 10, 2018 at 6:26 PM, Matt Burgess <mattyb149@apache.org> wrote:

> I'm fine with a vote, but I'll be voting to keep Java as the single
> language for the (non-test) code. I share the same concerns as many of
> the other folks as far as accepting other languages, it's mainly the
> "slippery slope" argument that I don't want to turn into a
> JVM-language flame war.  If Scala, why not Groovy? Certainly the
> syntax is closer to Java, and the community has accepted it as a valid
> language for writing unit tests, although we stopped short for
> allowing it for the deployable NiFi codebase, for the same reasons
> IIRC.  If Scala and/or Groovy, why not Kotlin?  The same argument
> (albeit more tenuous) goes for Clojure and just about every other JVM
> language (although I don't expect a call for LuaJ processors lol).
>
> Whether we decide to support various languages ad-hoc or not, I would
> strenuously object to multiple/hybrid build systems for the deployed
> artifacts. If I could switch NiFi completely to Gradle I would, but I
> realize there are good reasons for not doing so (yet?) in the Apache
> NiFi community, and I would never want any hybrid Maven/Gradle build
> for the deployable code, likewise for SBT, Leiningen, etc. With a
> custom Mojo for Maven NAR builds, and the complexity for hybrid builds
> in general, I think this would create a maintenance nightmare.
>
> The language thing is a tough decision though, it's not awesome that
> specifying a single language can be a barrier to a more diverse
> community, certainly Scala-based bundles would be more than welcome in
> the overall NiFi ecosystem, I just think the cons outweigh the pros
> for the baseline code. I've written Groovy processors/NARs using
> Gradle as the build system, and I'm good with keeping them in my own
> repo, especially when the Extension Registry becomes a thing. I can
> see the Extension Registry perhaps making this a moot point, but
> clearly we need to have this discussion in the meantime.
>
> Regards,
> Matt
>
>
> On Sat, Feb 10, 2018 at 5:23 PM, Andrew Grande <aperepel@gmail.com> wrote:
> > Wasn't there a warning trigger about the NiFi distro size from Apache
> > recently? IMO, before talking alternative languages, solve the modularity
> > and NAR distribution problem. I think the implementation of a module
> won't
> > matter much then, the point being not everything has to go in the core,
> > base distribution, but can still be easily sourced from a known repo, for
> > example.
> >
> > I have a feeling NiFi 1.6+ can be approaching 2GB distro size soon :)
> >
> > Andrew
> >
> > On Sat, Feb 10, 2018, 5:12 PM Joey Frazee <joey.frazee@icloud.com>
> wrote:
> >
> >> This probably necessitates a vote, yeah?
> >>
> >> Frankly, I’m usually happier writing Scala, and I’ve not encountered any
> >> problems using processors written in Scala, but I think it’ll be
> important
> >> to tread lightly.
> >>
> >> There’s a few things that pop into my head:
> >>
> >> - Maintainability and reviewability. A very very good Java developer
> need
> >> not, by definition, either know how to write or identify good Scala or
> spot
> >> problems and bugs.
> >> - Every Scala processor would either end up with a 5MB scala-lang.jar
> >> packaged into the .nar or we’d have to start including it in the core
> >> somewhere, if it’s not. It’s possible it might have already gotten
> pulled
> >> up from other dependencies.
> >> - Style. There’s a tremendous amount of variation in Scala style because
> >> of its type system, implicits, macros, and functional nature. There are
> >> very good people out there that can write good Scala that isn’t
> readable by
> >> the 99%.
> >> - Binary compatibility. Scala tends to be a little more brazen about
> >> breaking binary compatibility in major releases and those happen a bit
> more
> >> often than with Java. That’s not a problem for any potential source
> code in
> >> the project, but it could present some dependency issues someday.
> >> - Testing. There’s N > 1 test frameworks and testing styles within
> those,
> >> so there’s a lot of options for introducing more variability into the
> tests.
> >> - NiFi uses a lot of statics in setting up properties and relationships
> >> and the like, and idiomatic Scala approaches that stuff a bit
> differently,
> >> so it’ll be necessary to impose some style guidelines so there isn’t too
> >> much variation.
> >>
> >> That said, there are some things that won’t be problematic:
> >>
> >> - As mentioned, processors written in Scala do just work.
> >> - The scala-maven-plugin works just fine allowing mixed Java-Scala
> >> projects (btw, it’d probably be not super great to do mixed Java-Scala
> and
> >> mixed Maven-SBT though).
> >> - A lot of the above concerns could be addressed by having clear style
> >> guidelines.
> >>
> >> Another thing: most of the projects I see deliver separate jars for
> Scala
> >> components are delivering idiomatic APIs wrapping Java (or vice versa).
> I
> >> think publishing   a separate set of jars/nars for stuff written in
> Scala
> >> would be odd since here it’d mostly be processors with new functionality
> >> and not functionality for using Scala. I could imagine a lib of
> implicits,
> >> traits, classes that could make the Scala development more enjoyable.
> That
> >> probably would make sense to deliver that way.
> >>
> >> -joey
> >>
> >> On Feb 10, 2018, 10:33 AM -0600, Bryan Bende <bbende@gmail.com>, wrote:
> >> > I agree more with Andy about sticking with Java. The more varying
> >> languages
> >> > used, the more challenging it is to maintain. Once the code is part of
> >> the
> >> > Apache NiFi git repo, it is now the responsibility of the committers
> and
> >> > PMC members to maintain it.
> >> >
> >> > I’d even say I am somewhat against the groovy/Spock test code that
> Andy
> >> > mentioned. I have frequently spent hours trying to fix a Spock test
> that
> >> > broke from something I was working on. Every committer is familiar
> with
> >> > JUnit, but only a couple know Spock. Just using this as an example
> that
> >> > every committer knows Java, but only a couple probably know Scala,
> >> Clojure,
> >> > etc.
> >> >
> >> > On Sat, Feb 10, 2018 at 10:25 AM Jeff <jtswork@gmail.com> wrote:
> >> >
> >> > > +1 to Joe's response. If you can develop a component in Groovy or
> Scala
> >> > > (or Clojure!) more quickly/comfortably, or if allowing components
> >> written
> >> > > in other languages would encourage people to contribute more, I'm
> all
> >> for
> >> > > it.
> >> > >
> >> > > On Sat, Feb 10, 2018 at 7:42 AM Joe Witt <joe.witt@gmail.com>
> wrote:
> >> > >
> >> > > > i personally would be ok with it for an extension/processor
> provided
> >> it
> >> > > > integrates well with the build.
> >> > > >
> >> > > > i would agree with andys view for core framework stuff but for
> >> > > extensions i
> >> > > > think we can do it like mikethomsen suggested.
> >> > > >
> >> > > > others?
> >> > > >
> >> > > > thanks
> >> > > > joe
> >> > > >
> >> > > > On Feb 10, 2018 7:30 AM, "Mike Thomsen" <mikerthomsen@gmail.com>
> >> wrote:
> >> > > >
> >> > > > > I'm just a community contributor, so take that FWIW, but
a
> >> compromise
> >> > > > might
> >> > > > > be to publish the Scala code as separate maven modules to
maven
> >> central
> >> > > > and
> >> > > > > then submit a thoroughly tested processor written in Java.
As
> long
> >> as
> >> > > you
> >> > > > > have enough unit and integration tests to give strong coverage,
> I
> >> > > > wouldn't
> >> > > > > imagine anyone here would have issues reviewing it. If the
tests
> >> fail
> >> > > > > because of code issues in the external dependencies, the
obvious
> >> answer
> >> > > > is
> >> > > > > to just hold the PR until the tests pass.
> >> > > > >
> >> > > > > On Fri, Feb 9, 2018 at 9:00 AM, Weiss, Adam <
> >> > > Adam.Weiss@perkinelmer.com
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Devs,
> >> > > > > >
> >> > > > > > I have some interns starting with my team and we use
Scala
> >> internally
> >> > > > for
> >> > > > > > our work.
> >> > > > > > If I wanted to have them work to contribute some new
> processors,
> >> > > would
> >> > > > > > they have to be Java to be included with the base
> distribution or
> >> > > could
> >> > > > > > they use Scala as long as it fit within your current
build and
> >> test
> >> > > > > process?
> >> > > > > >
> >> > > > > > Thanks,
> >> > > > > > -Adam
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> > --
> >> > Sent from Gmail Mobile
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message