samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vadim Chekan <kot.bege...@gmail.com>
Subject Re: Running Samza programmatically
Date Tue, 29 Mar 2016 23:28:57 GMT
You can set
job.factory.class=org.apache.samza.job.local.ThreadJobFactory
samza property in config file and debug the job in IDE.

Vadym.


On Wed, Mar 9, 2016 at 3:47 AM, Robert Zuljevic <r.zuljevic@levi9.com>
wrote:

> Hi all,
>
>
>
> We would like to write and run integration tests for our own Samza
> Tasks/Jobs, but without using external tools like Zoopkio, but rather as a
> part of the build process. What we would like to do is start
> Yarn/Zookeeper/Kafka locally (or in a docker container) and then submit
> jobs programmatically (ie bypassing run-job.sh script) in order to have a
> bit more control over status and life of jobs. This is how we are trying to
> start jobs currently:
>
>
>
> CommandLine cl = new CommandLine();
>
> OptionSet options =
> cl.parser().parse("--config-factory=org.apache.samza.config.factories.PropertiesConfigFactory",
>
>
> "--config-path=" +
> App.class.getResource("/config/test.properties").getPath());
>
> Config config = new MapConfig(configMap);
>
> JobRunner runner = new JobRunner(cl.loadConfig(options));
>
>
>
> The job is submitted successfully, but it immediately fails and no logs
> are produced. The config we are using is the standard hello-samza one, with
> the caveat that yarn.package.path property is set to the absolute path
> where the packaged *.tar.gz file resides.
>
>
>
> I know that run-class.sh adds a lot of dependencies to the class-path and
> sets up some variables, should we try to mimic this behavior as well, or is
> there a more straightforward approach?
>
>
>
> Thank you for your response!
>
>
>
> Met vriendelijke groet / Kind regards,
>
> Robert Žuljević
>
> Software Developer
>
> [image: Title: Levi9 IT Services]
> ------------------------------
>
> Address: Trifkovicev trg 6, 21000 Novi Sad, Serbia
>
> Tel.: +31 20 6701 947 | +381 21 2155 500
>
> Mobile: +381 64 428 28 46
>
> Skype: robert.zuljevic
>
> Internet: www.levi9.com
>
>
>
> Chamber of commerce Levi9 Holding: 34221951
>
> Chamber of commerce Levi9 IT Services BV: 34224746
> ------------------------------
>
> This e-mail may contain confidential or privileged information. If you are
> not (one of) the intended recipient(s), please notify the sender
> immediately by reply e-mail and delete this message and any attachments
> permanently without retaining a copy. Any review, disclosure, copying,
> distribution or taking any action in reliance on the contents of this
> e-mail by persons or entities other than the intended recipient(s) is
> strictly prohibited and may be unlawful.
>
> The services of Levi9 are exclusively subject to its general terms and
> conditions. These general terms and conditions can be found on
> www.levi9.com and a copy will be promptly submitted to you on your
> request and free of charge.
>
>
>



-- 
>From RFC 2631: In ASN.1, EXPLICIT tagging is implicit unless IMPLICIT is
explicitly specified

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message