hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven van Beelen <smcvbee...@gmail.com>
Subject Re: BSP Task Input/InputSplit Filename
Date Tue, 21 May 2013 14:37:26 GMT
Thanks for the notes Chia-Hung Lin

I'm familiar to setting my own variables in the Configuration or retrieving
the directory with the Constant JOB_INPUT_DIR.
But what if you have 1+ files, in which each task will get it's own file?
How does a task know which file you set by:

    conf.set("my.path1", "/path/to/file1")
    conf.set("my.path2", "/path/to/file2")
    ....


is the one it is working one? Or am I missing something trivial here?
That is what I'm trying to figure out.


On Tue, May 21, 2013 at 4:27 PM, Chia-Hung Lin <clin4j@googlemail.com>wrote:

> My understanding is that you can configure path during constructing a job.
>
>     HamaConfiguration conf = new HamaConfiguration();
>     conf.set("my.path","/path/to/file")
>     BSPJob bsp = new BSPJob(conf, MyBSP.class);
>
> And wihtin customized BSP class e.g. MyBSP calls
>
>     BSPPeer.getConfiguration();
>
> to retrieve the file name.
>
> Or FileInputFormat makes use of Configuration set input path with the
> key "bsp.input.dir". Path should be able to obtain using
> conf.get("bsp.input.dir") when performing computation.
>
>
>
>
>
>
>
>
>
>
>
> On 21 May 2013 18:04, Steven van Beelen <smcvbeelen@gmail.com> wrote:
> > Hi all,
> >
> > The title says it: is there a way to retrieve the filename of the
> > input/inputsplit a BSP Task is working on? I've been looking for some
> time
> > in the docs and source files, but cannot seem to find if one is able to
> > retrieve the filename/pathname from the input used.
> >
> > Cheers
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message