storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From János Háber <janos.ha...@finesolution.hu>
Subject Re: Sample project using Storm and Kafka (using Scala 2.10)
Date Thu, 22 May 2014 07:19:36 GMT
I understand this, but before you create a version in central maven
repository from storm-kafka project please correct the build or add the
_2.9 prefix to the end of the jar.

b0c1

ps: if you want and the apache build server support, I can create correct
project build in sbt (I never used gradle)...

János Háber
Fine Solution Ltd



On Thu, May 22, 2014 at 3:43 AM, P. Taylor Goetz <ptgoetz@gmail.com> wrote:

> János,
>
> I sense and appreciate your frustration.
>
> The initial goal with bringing the storm-kafka project under the Apache
> Storm project umbrella is to provide a definitive source for the code and
> eliminate fragmentation, which has been an issue in the past. This is a
> good first step for many reasons, and will hopefully improve greatly in
> future iterations.
>
> Just to be clear, Apache releases source code, not binaries. Any binaries
> built against an Apache project's source are provided as a convenience.
>
> The cross-compilation ballet you describe is a feature of scala [1], not
> anything Storm or maven related. Yes, we can and will improve on the build
> and binary release process. But for now the goal is to provide a definitive
> source, and make sure users can build that source as needed -- which I
> think we have done.
>
> -Taylor
>
> [1] a not-so-subtle dig against scala's build process, not the language.
> Hopefully this will get sorted out someday.
>
> On May 21, 2014, at 7:48 PM, János Háber <janos.haber@finesolution.hu>
> wrote:
>
> Dear Taylor, I love your work, but:
> - I don't want to build myself
> - Dependent libraries (like tormenta-kafka) need a cross compiled version
> of storm-kafka, without this they need to clone the project, change the
> group id, handle every changes by hand, and publish to central repository.
> - I need to have own maven repository to store the cross compiled version
> (which need to be public if somebody want to use my application) and
> maintain the changes
> - I think hand made library is the best way to make a project to unstable,
> because if I clone the project I need to clone tormenta-kafka project too
> and handle myself both version changes, solve the compatibility issues,
> etc...
>
> I know how can I compile to 2.10 by hand, but all other project (example
> kafka, which is apache project too) has cross compiled version in CENTRAL
> maven repository, if a project required a cross-compiled scala library - my
> oppinion - the project need to provide cross-compiled version too, no
> exception.
>
> b0c1
>
> János Háber
> Fine Solution Ltd
>
>
>
> On Wed, May 21, 2014 at 11:48 PM, P. Taylor Goetz <ptgoetz@gmail.com>wrote:
>
>> If you build yourself, you can do the following:
>>
>> mvn install -DscalaVersion=2.10.3 -DkafkaArtifact=kafka_2.10
>>
>> - Taylor
>>
>>
>> On May 21, 2014, at 5:32 PM, János Háber <janos.haber@finesolution.hu>
>> wrote:
>>
>>
>> https://github.com/apache/incubator-storm/blob/master/external/storm-kafka/pom.xml#L34
>> And 2.10 not backward compatible with 2.9... that's why scala
>> applications/libraries is cross compiled with multiple scala version.
>> (that's why SBT will handle this natively, but I think in apache sbt is not
>> allowed so you need to create multiple maven project (hard way) or switch
>> to gradle (like kafka) to produce multiple version)
>>
>> János Háber
>> Fine Solution Ltd
>>
>>
>>
>> On Wed, May 21, 2014 at 11:28 PM, János Háber <
>> janos.haber@finesolution.hu> wrote:
>>
>>>
>>> On Wed, May 21, 2014 at 11:27 PM, P. Taylor Goetz <ptgoetz@gmail.com>wrote:
>>>
>>>> ill include the Kafka spout (for Kafka 0.8.x).
>>>>
>>>
>>> Yeah, but the current maven build compile to 2.9 scala not 2.10...
>>>
>>>
>>> János Háber
>>> Fine Solution Ltd
>>>
>>>
>>
>>
>

Mime
View raw message