samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@linkedin.com>
Subject Re: IDE Setup
Date Wed, 11 Dec 2013 17:19:28 GMT
+ dev@…

Hey Klaus,

You do not need to build Samza jobs using Maven. Any Java build system will do. The hello-samza
project simply uses Maven because it's the most well known/widely adopted Java build system
out there right now. Internally, at LinkedIn, we use Gradle, for example.

There is a "local mode" for Samza as well. Samza jobs can be executed using the "LocalJobFactory"
instead of the "YarnJobFactory". This will start a single Samza container that processes all
input messages locally on your development machine. To enable the LocalJobFactory, change
this line in your Samza job's config:

  job.factory.class=org.apache.samza.job.yarn.YarnJobFactory

To:

  job.factory.class=org.apache.samza.job.local.LocalJobFactory

Then run the normal run-job.sh script. You'll see that all STDOUT is now visible on your console,
and a log file is created in $PWD that has all log4j output in it.

There is also this tutorial on enabling debugging for your Samza job:

  http://samza.incubator.apache.org/learn/tutorials/0.7.0/remote-debugging-samza.html

Cheers,
Chris

From: Klaus Schaefers <klaus.schaefers@ligatus.com<mailto:klaus.schaefers@ligatus.com>>
Date: Wednesday, December 11, 2013 2:00 AM
To: Chris Riccomini <criccomini@linkedin.com<mailto:criccomini@linkedin.com>>
Subject: Re: IDE Setup

Hi,

thanks this worked and I could import the files into IntelliJ. However I have some questions
with regards to the suggested development workflow. From my understanding I have to build
my streaming app with maven and deploy it manually to YARN. So every time I do a mayor change
I would to rerun all the maven command and deploy again to YARN. Is there any more comfortable
way of development? Storm has it's local mode and hadoop has a testing framework. Is there
something similar in SAMZA?


Cheers,

Klaus




On Wed, Dec 11, 2013 at 10:05 AM, Klaus Schaefers <klaus.schaefers@ligatus.com<mailto:klaus.schaefers@ligatus.com>>
wrote:
Hi,

thanks for the answer!

Cheers,

Klaus



On Tue, Dec 10, 2013 at 8:06 PM, Chris Riccomini <criccomini@linkedin.com<mailto:criccomini@linkedin.com>>
wrote:
Hey Klaus,

I'm assuming you're trying to work with hello-samza
(https://github.com/linkedin/hello-samza). If you're using Eclipse, you
can run:

  mvn eclipse:eclipse

>From the hello-samza directory. Once that command finishes, you can go to
Eclipse, and click File > Import > General > Existing Projects into
Workspace > Next > Select root directory > Browse > (navigate to
hello-samza folder) > Open > Finish. This will load all the hello-samza
projects into Eclipse.

If you're running IntelliJ, you should be able to import the project using
IntelliJ's Maven support:


http://wiki.jetbrains.net/intellij/Creating_and_importing_Maven_projects#Im
porting_an_existing_Maven_project_into_an_IntelliJ_IDEA_project

If you're interested in looking at Samza's code
(https://git-wip-us.apache.org/repos/asf?p=incubator-samza.git;a=tree)
instead of hello-samza, you check out samza, and run:

  ./gradlew eclipse

For Eclipse. Then, simply follow the same import process described above.

For IntelliJ, you'd run:

  ./gradlew idea

And then import the project into IntelliJ:


http://www.jetbrains.com/idea/webhelp/creating-a-project-by-importing-exist
ing-sources.html<http://www.jetbrains.com/idea/webhelp/creating-a-project-by-importing-existing-sources.html>

Cheers,
Chris

On 12/10/13 4:02 AM, "Klaus Schaefers" <klaus.schaefers@ligatus.com<mailto:klaus.schaefers@ligatus.com>>
wrote:

>Hi,can somebody give my a short hint how to setup my IDE to write my own
>Samza jobs? Sorry I am new to maven and I have so troubles including the
>decencies.
>
>
>Cheers,
>
>Klaus
>
>--
>
>--
>
>Klaus Schaefers
>Senior Optimization Manager
>
>Ligatus GmbH
>Hohenstaufenring 30-32
>D-50674 Köln
>
>Tel.:  +49 (0) 221 / 56939 -784<tel:%2B49%20%280%29%20221%20%2F%2056939%20-784>
>Fax:  +49 (0) 221 / 56 939 - 599<tel:%2B49%20%280%29%20221%20%2F%2056%20939%20-%20599>
>E-Mail: klaus.schaefers@ligatus.com<mailto:klaus.schaefers@ligatus.com>
>Web: www.ligatus.de<http://www.ligatus.de>
>
>HRB Köln 56003
>Geschäftsführung:
>Dipl.-Kaufmann Lars Hasselbach, Dipl.-Kaufmann Klaus Ludemann,
>Dipl.-Wirtschaftsingenieur Arne Wolter




--

--

Klaus Schaefers
Senior Optimization Manager

Ligatus GmbH
Hohenstaufenring 30-32
D-50674 Köln

Tel.:  +49 (0) 221 / 56939 -784<tel:%2B49%20%280%29%20221%20%2F%2056939%20-784>
Fax:  +49 (0) 221 / 56 939 - 599<tel:%2B49%20%280%29%20221%20%2F%2056%20939%20-%20599>
E-Mail: klaus.schaefers@ligatus.com<mailto:klaus.schaefers@ligatus.com>
Web: www.ligatus.de<http://www.ligatus.de/>

HRB Köln 56003
Geschäftsführung:
Dipl.-Kaufmann Lars Hasselbach, Dipl.-Kaufmann Klaus Ludemann, Dipl.-Wirtschaftsingenieur
Arne Wolter



--

--

Klaus Schaefers
Senior Optimization Manager

Ligatus GmbH
Hohenstaufenring 30-32
D-50674 Köln

Tel.:  +49 (0) 221 / 56939 -784
Fax:  +49 (0) 221 / 56 939 - 599
E-Mail: klaus.schaefers@ligatus.com<mailto:klaus.schaefers@ligatus.com>
Web: www.ligatus.de<http://www.ligatus.de/>

HRB Köln 56003
Geschäftsführung:
Dipl.-Kaufmann Lars Hasselbach, Dipl.-Kaufmann Klaus Ludemann, Dipl.-Wirtschaftsingenieur
Arne Wolter

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message