samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@apache.org>
Subject Re: Submitting yarn job with custom properties
Date Tue, 24 Mar 2015 20:27:17 GMT
FYI- I haven't actually tried this, but I think it should work. If
__package doesn't work, try without the __package/ prefix (i.e.
fopen('yourfile.txt')).

On Tue, Mar 24, 2015 at 1:26 PM, Chris Riccomini <criccomini@apache.org>
wrote:

> Cool, that should work. In YARN, the .tgz file is unzipped into
> $CWD/__package/, so if you have a file in the root of your tarball, then
> fopen('__package/yourfile.txt') should work.
>
> On Tue, Mar 24, 2015 at 1:22 PM, Shekar Tippur <ctippur@gmail.com> wrote:
>
>> Thanks Chris. I was trying option #2.
>>
>> - Shekar
>>
>> On Tue, Mar 24, 2015 at 11:09 AM, Chris Riccomini <criccomini@apache.org>
>> wrote:
>>
>> > Hey Shekar,
>> >
>> > So you have two properties files? One is the Samza job file, and the
>> other
>> > is one with the format you described above?
>> >
>> > Several options:
>> >
>> > 1. Move these properties into the main Samza job file. You'll then have
>> > access to them via the Config object.
>> > 2. Put the property file into your .tgz file. You'll then have access
>> to it
>> > relative to the current working directory, from your code.
>> > 3. Put it somewhere remote (HDFS, HTTP, etc), and have your job pull it
>> > from there.
>> >
>> > Cheers,
>> > Chris
>> >
>> > On Mon, Mar 23, 2015 at 4:10 PM, Shekar Tippur <ctippur@gmail.com>
>> wrote:
>> >
>> > > I would like to decouple Samza properties with the custom ones (if
>> > > possible).
>> > >
>> > > - Shekar
>> > >
>> > > On Mon, Mar 23, 2015 at 3:12 PM, Shekar Tippur <ctippur@gmail.com>
>> > wrote:
>> > >
>> > > > Hello,
>> > > >
>> > > > I have a custom properties file with name value pairs.
>> > > >
>> > > > name1 value1
>> > > > name2 value2
>> > > > name3 value3
>> > > >
>> > > > I want to pass this to Yarn job. What is the best way to achieve
>> this?
>> > > > This works well when we declare locally or thread job.
>> > > >
>> > > > - Shekar
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message