airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Klosowski <dav...@thinknear.com>
Subject Re: DAGs not running
Date Fri, 22 Sep 2017 19:04:10 GMT
Thanks Bolke,

That's good to know.  We were looking into upgrading, so this gives us more
motivation ;-)

Cheers,
David


On Fri, Sep 22, 2017 at 11:57 AM, Bolke de Bruin <bdbruin@gmail.com> wrote:

> Please upgrade to 1.8.2. The were some ‘gaps’ in the 1.8.0 scheduler.
>
> Cheers
> Bolke
>
> > On 22 Sep 2017, at 19:54, David Klosowski <davidk@thinknear.com> wrote:
> >
> > Hey Airflow Devs:
> >
> > We noticed an issue with task instances not running sometimes.
> >
> > Some interesting details are:
> >> in the database it shows that specific tasks that are "stuck" have an
> > end_date but no state (null state)
> >> Airflow has been running for several days just fine and there were no
> > changes to Airflow in that period.
> >> it happened for two separate important tasks for us at the same time
> >
> > We're running:
> >> Airflow 1.8.0 with a Postgres backend
> >> Celery with Redis backed by Postgres result_backend
> >> 3 worker hosts and 1 scheduler host
> >
> > Let me know if you guys have any suggestions or advice.  Going to look
> > through the code to see if I can see where/how this might be happening,
> but
> > any suggestions or well known issues that might point to the problem
> might
> > be helpful.
> >
> > Thanks.
> >
> > Regards,
> > David
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message