nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Grande <apere...@gmail.com>
Subject Re: Will you accept contributions in Scala?
Date Sat, 10 Feb 2018 22:23:00 GMT
Wasn't there a warning trigger about the NiFi distro size from Apache
recently? IMO, before talking alternative languages, solve the modularity
and NAR distribution problem. I think the implementation of a module won't
matter much then, the point being not everything has to go in the core,
base distribution, but can still be easily sourced from a known repo, for
example.

I have a feeling NiFi 1.6+ can be approaching 2GB distro size soon :)

Andrew

On Sat, Feb 10, 2018, 5:12 PM Joey Frazee <joey.frazee@icloud.com> wrote:

> This probably necessitates a vote, yeah?
>
> Frankly, I’m usually happier writing Scala, and I’ve not encountered any
> problems using processors written in Scala, but I think it’ll be important
> to tread lightly.
>
> There’s a few things that pop into my head:
>
> - Maintainability and reviewability. A very very good Java developer need
> not, by definition, either know how to write or identify good Scala or spot
> problems and bugs.
> - Every Scala processor would either end up with a 5MB scala-lang.jar
> packaged into the .nar or we’d have to start including it in the core
> somewhere, if it’s not. It’s possible it might have already gotten pulled
> up from other dependencies.
> - Style. There’s a tremendous amount of variation in Scala style because
> of its type system, implicits, macros, and functional nature. There are
> very good people out there that can write good Scala that isn’t readable by
> the 99%.
> - Binary compatibility. Scala tends to be a little more brazen about
> breaking binary compatibility in major releases and those happen a bit more
> often than with Java. That’s not a problem for any potential source code in
> the project, but it could present some dependency issues someday.
> - Testing. There’s N > 1 test frameworks and testing styles within those,
> so there’s a lot of options for introducing more variability into the tests.
> - NiFi uses a lot of statics in setting up properties and relationships
> and the like, and idiomatic Scala approaches that stuff a bit differently,
> so it’ll be necessary to impose some style guidelines so there isn’t too
> much variation.
>
> That said, there are some things that won’t be problematic:
>
> - As mentioned, processors written in Scala do just work.
> - The scala-maven-plugin works just fine allowing mixed Java-Scala
> projects (btw, it’d probably be not super great to do mixed Java-Scala and
> mixed Maven-SBT though).
> - A lot of the above concerns could be addressed by having clear style
> guidelines.
>
> Another thing: most of the projects I see deliver separate jars for Scala
> components are delivering idiomatic APIs wrapping Java (or vice versa). I
> think publishing   a separate set of jars/nars for stuff written in Scala
> would be odd since here it’d mostly be processors with new functionality
> and not functionality for using Scala. I could imagine a lib of implicits,
> traits, classes that could make the Scala development more enjoyable. That
> probably would make sense to deliver that way.
>
> -joey
>
> On Feb 10, 2018, 10:33 AM -0600, Bryan Bende <bbende@gmail.com>, wrote:
> > I agree more with Andy about sticking with Java. The more varying
> languages
> > used, the more challenging it is to maintain. Once the code is part of
> the
> > Apache NiFi git repo, it is now the responsibility of the committers and
> > PMC members to maintain it.
> >
> > I’d even say I am somewhat against the groovy/Spock test code that Andy
> > mentioned. I have frequently spent hours trying to fix a Spock test that
> > broke from something I was working on. Every committer is familiar with
> > JUnit, but only a couple know Spock. Just using this as an example that
> > every committer knows Java, but only a couple probably know Scala,
> Clojure,
> > etc.
> >
> > On Sat, Feb 10, 2018 at 10:25 AM Jeff <jtswork@gmail.com> wrote:
> >
> > > +1 to Joe's response. If you can develop a component in Groovy or Scala
> > > (or Clojure!) more quickly/comfortably, or if allowing components
> written
> > > in other languages would encourage people to contribute more, I'm all
> for
> > > it.
> > >
> > > On Sat, Feb 10, 2018 at 7:42 AM Joe Witt <joe.witt@gmail.com> wrote:
> > >
> > > > i personally would be ok with it for an extension/processor provided
> it
> > > > integrates well with the build.
> > > >
> > > > i would agree with andys view for core framework stuff but for
> > > extensions i
> > > > think we can do it like mikethomsen suggested.
> > > >
> > > > others?
> > > >
> > > > thanks
> > > > joe
> > > >
> > > > On Feb 10, 2018 7:30 AM, "Mike Thomsen" <mikerthomsen@gmail.com>
> wrote:
> > > >
> > > > > I'm just a community contributor, so take that FWIW, but a
> compromise
> > > > might
> > > > > be to publish the Scala code as separate maven modules to maven
> central
> > > > and
> > > > > then submit a thoroughly tested processor written in Java. As long
> as
> > > you
> > > > > have enough unit and integration tests to give strong coverage, I
> > > > wouldn't
> > > > > imagine anyone here would have issues reviewing it. If the tests
> fail
> > > > > because of code issues in the external dependencies, the obvious
> answer
> > > > is
> > > > > to just hold the PR until the tests pass.
> > > > >
> > > > > On Fri, Feb 9, 2018 at 9:00 AM, Weiss, Adam <
> > > Adam.Weiss@perkinelmer.com
> > > > > wrote:
> > > > >
> > > > > > Devs,
> > > > > >
> > > > > > I have some interns starting with my team and we use Scala
> internally
> > > > for
> > > > > > our work.
> > > > > > If I wanted to have them work to contribute some new processors,
> > > would
> > > > > > they have to be Java to be included with the base distribution
or
> > > could
> > > > > > they use Scala as long as it fit within your current build and
> test
> > > > > process?
> > > > > >
> > > > > > Thanks,
> > > > > > -Adam
> > > > > >
> > > > >
> > > >
> > >
> > --
> > Sent from Gmail Mobile
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message