airavata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcus Christie (JIRA)" <>
Subject [jira] [Commented] (AIRAVATA-2352) Orchestrator sometimes stops processing messages from experiment_launch queue
Date Wed, 29 Mar 2017 13:33:42 GMT


Marcus Christie commented on AIRAVATA-2352:

For the gw153 event on 3/29, the last time it processed an experiment_launch event was at
9:55 AM on 3/27
2017-03-27 09:55:44 [pool-15-thread-3] INFO o.a.a.m.c.impl.ExperimentConsumer -  Message Received
with message id 'LAUN
CH.EXP-7c289ece-afb9-4930-b725-754d092fc2bb' and with message type 'EXPERIMENT'  for experimentId:

The last process status change event it processed was at 11:15am on 3/27
2017-03-27 11:15:35 [pool-13-thread-3] INFO o.a.a.o.s.OrchestratorServerHandler - expId: US3-AIRA_45911bfe-ea37-4f18-8fc5-a4b182709ff9,
processId: PROCESS_a32f0fbd-1196-4682-8c5a-0081365ffb93 :- Process status changed event received
for status COMPLETED

The logs show no errors. No further events are processed until the server was restarted at
about 9:11am on 3/29.  When the server was restarted there were 61 messages in the experiment_launch

> Orchestrator sometimes stops processing messages from experiment_launch queue
> -----------------------------------------------------------------------------
>                 Key: AIRAVATA-2352
>                 URL:
>             Project: Airavata
>          Issue Type: Bug
>    Affects Versions: 0.17
>            Reporter: Marcus Christie
>            Priority: Critical
> This was observed on 3/29 9am for
> *Workaround*: restart the api-orch server. Once that is done the orchestrator starts
picking up events from experiment_launch again.
> I'm not seeing anything in the logs to indicate why the orchestrator stopped processing
experiment_launch events.

This message was sent by Atlassian JIRA

View raw message