samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@apache.org>
Subject Re: Submitting yarn job with custom properties
Date Tue, 24 Mar 2015 20:26:16 GMT
Cool, that should work. In YARN, the .tgz file is unzipped into
$CWD/__package/, so if you have a file in the root of your tarball, then
fopen('__package/yourfile.txt') should work.

On Tue, Mar 24, 2015 at 1:22 PM, Shekar Tippur <ctippur@gmail.com> wrote:

> Thanks Chris. I was trying option #2.
>
> - Shekar
>
> On Tue, Mar 24, 2015 at 11:09 AM, Chris Riccomini <criccomini@apache.org>
> wrote:
>
> > Hey Shekar,
> >
> > So you have two properties files? One is the Samza job file, and the
> other
> > is one with the format you described above?
> >
> > Several options:
> >
> > 1. Move these properties into the main Samza job file. You'll then have
> > access to them via the Config object.
> > 2. Put the property file into your .tgz file. You'll then have access to
> it
> > relative to the current working directory, from your code.
> > 3. Put it somewhere remote (HDFS, HTTP, etc), and have your job pull it
> > from there.
> >
> > Cheers,
> > Chris
> >
> > On Mon, Mar 23, 2015 at 4:10 PM, Shekar Tippur <ctippur@gmail.com>
> wrote:
> >
> > > I would like to decouple Samza properties with the custom ones (if
> > > possible).
> > >
> > > - Shekar
> > >
> > > On Mon, Mar 23, 2015 at 3:12 PM, Shekar Tippur <ctippur@gmail.com>
> > wrote:
> > >
> > > > Hello,
> > > >
> > > > I have a custom properties file with name value pairs.
> > > >
> > > > name1 value1
> > > > name2 value2
> > > > name3 value3
> > > >
> > > > I want to pass this to Yarn job. What is the best way to achieve
> this?
> > > > This works well when we declare locally or thread job.
> > > >
> > > > - Shekar
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message