Thanks for the notes Chia-Hung Lin
I'm familiar to setting my own variables in the Configuration or retrieving
the directory with the Constant JOB_INPUT_DIR.
But what if you have 1+ files, in which each task will get it's own file?
How does a task know which file you set by:
conf.set("my.path1", "/path/to/file1")
conf.set("my.path2", "/path/to/file2")
....
is the one it is working one? Or am I missing something trivial here?
That is what I'm trying to figure out.
On Tue, May 21, 2013 at 4:27 PM, Chia-Hung Lin <clin4j@googlemail.com>wrote:
> My understanding is that you can configure path during constructing a job.
>
> HamaConfiguration conf = new HamaConfiguration();
> conf.set("my.path","/path/to/file")
> BSPJob bsp = new BSPJob(conf, MyBSP.class);
>
> And wihtin customized BSP class e.g. MyBSP calls
>
> BSPPeer.getConfiguration();
>
> to retrieve the file name.
>
> Or FileInputFormat makes use of Configuration set input path with the
> key "bsp.input.dir". Path should be able to obtain using
> conf.get("bsp.input.dir") when performing computation.
>
>
>
>
>
>
>
>
>
>
>
> On 21 May 2013 18:04, Steven van Beelen <smcvbeelen@gmail.com> wrote:
> > Hi all,
> >
> > The title says it: is there a way to retrieve the filename of the
> > input/inputsplit a BSP Task is working on? I've been looking for some
> time
> > in the docs and source files, but cannot seem to find if one is able to
> > retrieve the filename/pathname from the input used.
> >
> > Cheers
>
|