whirr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrei Savu <savu.and...@gmail.com>
Subject Re: launching a one job cluster
Date Thu, 22 Sep 2011 22:09:41 GMT
Why is so important to avoid having a poll? The cost is low and almost
any job is running at least for a few minutes.

-- Andrei

On Thu, Sep 22, 2011 at 3:07 PM, David Alves <davidralves@gmail.com> wrote:
> Hi Andrei
>
>        I know…
>        The thing is that code used the Hadoop JobClient class's runJob() method that
actually polls for progress.
>        I am not using hadoop (in hindsight using the word "job" might have been a
mistake) and I was wondering if there is already a way to do that for generic cases (e.g.,
scripts or java programs).
>        In particular as I'm collecting accurate metrics I'd like a non poll based
technique.
>        Even if there is none I can always try and code it, so all ideas are welcome.
>
> thanks
> david
>
>
> On Sep 22, 2011, at 4:52 PM, Andrei Savu wrote:
>
>> This is exactly what the example code is doing (and the hadoop
>> integration test). The job running code is blocking while the job is
>> executing.
>>
>> -- Andrei Savu / andreisavu.ro
>>
>> On Thu, Sep 22, 2011 at 2:03 PM, David Alves <davidralves@gmail.com> wrote:
>>> Hi All
>>>
>>>        I need to launch a cluster run a job and terminate the cluster as
the job is finished (as soon as possible).
>>>        Is there any "nice" way to do this, or do you have any suggestions?
>>>        On the top of my head I can imagine some quick and dirty solutions
(like creating a file whenever the task is completed and polling for its existence from the
whirr handler) but I'd like to do it without polling if possible. Any ideas?
>>>
>>> thanks
>>> -david
>>>
>
>

Mime
View raw message