uima-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eddie Epstein <eaepst...@gmail.com>
Subject Re: uima ducc wait_for_completion
Date Mon, 22 Feb 2016 22:59:34 GMT
Hi Yi-Wen,

Here is an example console from a normal DUCC job, with comments

2016.02.19 10:33:34 Fri Received
2016.02.19 10:33:34 Fri WaitingForDriver          ... waiting for the Job
Driver to initialize successfully
2016.02.19 10:33:52 Fri WaitingForResources   ... waiting for resources to
start one or more Job Process (pipelines)
2016.02.19 10:34:00 Fri Assigned
2016.02.19 10:34:02 Fri Initializing                     ... one or more JP
launched and doing initialization
2016.02.19 10:35:23 Fri Running                        ... initialization
completed successfully, processing work items from JD
2016.02.19 10:36:14 Fri Completing                   ... job processes are
terminating
2016.02.19 10:36:31 Fri Completed                    ... all user processes
have terminated
2016.02.19 10:36:31 Fri completion type: EndOfJob
2016.02.19 10:36:31 Fri rationale: state manager detected normal completion
   ... all work items processed

Your console indicates that the job driver was terminated before
initialization completed.
There should be a lot more information in the job's log directory. Feel
free to zip up and email that data to Lou or I, as long as it does *not*
contain any confidential information.

Regards,
Eddie


On Mon, Feb 22, 2016 at 5:18 PM, Yi-Wen Liu <yiwenliu@usc.edu> wrote:

> Hi,
>
> The monitor output from ducc.log:
>
> 2015.11.30 02:27:16 Mon Received
> 2015.11.30 02:27:16 Mon WaitingForDriver
> 2015.11.30 02:27:44 Mon WaitingForResources
> 2015.11.30 02:27:56 Mon Assigned
> 2015.11.30 02:28:04 Mon Initializing
> 2015.11.30 02:29:35 Mon Completing
> 2015.11.30 02:30:18 Mon Completed
> 2015.11.30 02:30:18 Mon completion type: CanceledByDriver
> 2015.11.30 02:30:18 Mon rationale: unspecified
>
>
>
> Thanks,
> Yi-Wen
>
> On Sun, Feb 21, 2016 at 1:45 PM, Eddie Epstein <eaepstein@gmail.com>
> wrote:
>
> > Hi,
> > Can you please paste the full monitor output from the DUCC job submit
> here?
> > Thanks,
> > Eddie
> >
> > On Sun, Feb 21, 2016 at 3:26 PM, Yi-Wen Liu <yiwenliu@usc.edu> wrote:
> >
> > > Hi,
> > >
> > > Because when I used wait_for_completion, I saw the terminal print out
> the
> > > step by step state, and after it finished, the output file was not
> > > generated yet. The job status on web server page might still stay in
> > > "waiting for resources" instead of completion.
> > >
> > > And sorry for the confusion, I used "main process" to refer to the
> > process
> > > generating output (the second one below).
> > >
> > > Time axis---->
> > > *The terminal*                  status:waiting for driver -> ....->
> > status:
> > > completed (point A, no output)
> > > *The web server page    *initializing -> waiting for
> > >
> > >
> >
> resouces............................................................->completion
> > > (point B, has output)
> > >
> > > And my question is when can I know the point when the output is
> generated
> > > (point B)?
> > > If I use wait_for_completion, it is telling me the point A but there is
> > no
> > > output yet.
> > >
> > > If I misunderstand anything please correct me, thanks for your
> patience!
> > >
> > > Thanks,
> > > Yi-Wen
> > >
> > > On Sun, Feb 21, 2016 at 6:05 AM, Lou DeGenaro <lou.degenaro@gmail.com>
> > > wrote:
> > >
> > > > Yi-Wen,
> > > >
> > > > When submitting a DUCC Job, wait for completion should prevent the
> > return
> > > > until all your work items have been processed.
> > > >
> > > > What exactly do you mean by "main process"?
> > > >
> > > > Lou.
> > > >
> > > > On Sat, Feb 20, 2016 at 10:30 PM, Yi-Wen Liu <yiwenliu@usc.edu>
> wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > I am using ducc in a web app, and I need to know when does ducc
> > finish
> > > > its
> > > > > work, ie, generate the output file.
> > > > > I use the two properties in my job file:
> > > > >
> > > > > cancel_on_interrupt
> > > > > wait_for_completion
> > > > >
> > > > > However, when I submitted job file, it did show the status on
> > terminal,
> > > > but
> > > > > it finished earlier than the main process, I have to wait for few
> > > minutes
> > > > > for the output file to be generated.
> > > > >
> > > > > Is there any way to get notified when the main process finishes
> > without
> > > > > looking at the web server page?
> > > > >
> > > > > Any suggestion is appreciated!
> > > > >
> > > > >
> > > > > Thanks,
> > > > > Yi-Wen
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message