airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From siddharth anand <san...@apache.org>
Subject Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc3
Date Mon, 13 Feb 2017 19:51:20 GMT
Folks!
I need to change my vote.. -1 (Binding).


Mark Success/Clear is broken in the UI. It's a regression.

-s

On Mon, Feb 13, 2017 at 10:53 AM, Alex Van Boxel <alex@vanboxel.be> wrote:

> +1 (binding)
>
> On Mon, Feb 13, 2017 at 7:45 PM siddharth anand <sanand@apache.org> wrote:
>
> > +1 (binding)
> >
> > On Mon, Feb 13, 2017 at 8:57 AM, Chris Riccomini <criccomini@apache.org>
> > wrote:
> >
> > > +1 (binding)
> > >
> > > On Sun, Feb 12, 2017 at 8:54 AM, Jeremiah Lowin <jlowin@apache.org>
> > wrote:
> > >
> > > > Interesting -- I also run on Kubernetes with a git-sync sidecar, but
> > the
> > > > containers wait for the synced repo to apprar before starting since
> it
> > > > contains some dependencies -- I assume that's why I didn't experience
> > the
> > > > same issue.
> > > >
> > > > On Sun, Feb 12, 2017 at 6:29 AM Bolke de Bruin <bdbruin@gmail.com>
> > > wrote:
> > > >
> > > > > Although the race condition doesn't explain why “num_runs = None”
> > > > resolved
> > > > > the issue for you earlier, but it does give a clue now: the PR that
> > > > > introduced “num_runs = -1” was there to be able to work with
empty
> > dag
> > > > > dirs, maybe it wasn’t fully covered yet.
> > > > >
> > > > > Bolke
> > > > >
> > > > > > On 12 Feb 2017, at 12:26, Bolke de Bruin <bdbruin@gmail.com>
> > wrote:
> > > > > >
> > > > > > Ok great! Thanks! That sounds like a race condition: module
not
> > > > > available yet at time of reading. I would expect that it resolves
> > > itself
> > > > > after a while.
> > > > > >
> > > > > > After talking to some people at the Warsaw BigData conf I have
> some
> > > > > ideas around syncing dags, Spoiler: no dependency on git.
> > > > > >
> > > > > > - Bolke
> > > > > >
> > > > > >> On 12 Feb 2017, at 11:17, Alex Van Boxel <alex@vanboxel.be>
> > wrote:
> > > > > >>
> > > > > >> Running ok, in staging... @bolke I'm running patch-less.
I've
> > > switched
> > > > > my
> > > > > >> Kubernetes from:
> > > > > >>
> > > > > >> - each container (webserver/scheduler/worker) had a git-sync'er
> > > > (getting
> > > > > >> the dags from git)
> > > > > >>> this meant that the scheduler had 0 dags at startup,
and should
> > > have
> > > > > >> picked them up later
> > > > > >>
> > > > > >> to
> > > > > >>
> > > > > >> - single NFS share that shares airflow_home over each container
> > > > > >>> the git sync'er is now a seperate container running
before the
> > > other
> > > > > >> containers
> > > > > >>
> > > > > >> This resolved my mystery DAG crashes.
> > > > > >>
> > > > > >> I'll be updating production to a patchless RC3 today, you
get my
> > > vote
> > > > > after
> > > > > >> that.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Sun, Feb 12, 2017 at 4:59 AM Boris Tyukin <
> > boris@boristyukin.com
> > > >
> > > > > wrote:
> > > > > >>
> > > > > >>> awesome! thanks Jeremiah
> > > > > >>>
> > > > > >>> On Sat, Feb 11, 2017 at 12:53 PM, Jeremiah Lowin <
> > > jlowin@apache.org>
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>>> Boris, I submitted a PR to address your second point
--
> > > > > >>>> https://github.com/apache/incubator-airflow/pull/2068.
> Thanks!
> > > > > >>>>
> > > > > >>>> On Sat, Feb 11, 2017 at 10:42 AM Boris Tyukin <
> > > > boris@boristyukin.com>
> > > > > >>>> wrote:
> > > > > >>>>
> > > > > >>>>> I am running LocalExecutor and not doing crazy
things but use
> > DAG
> > > > > >>>>> generation heavily - everything runs fine as
before. As I
> > > mentioned
> > > > > in
> > > > > >>>>> other threads only had a few issues:
> > > > > >>>>>
> > > > > >>>>> 1) had to upgrade MySQL which was a PAIN. Cloudera
CDH is
> > running
> > > > old
> > > > > >>>>> version of MySQL which was compatible with 1.7.1
but not
> > > compatible
> > > > > now
> > > > > >>>>> with 1.8 because of fractional seconds support
PR.
> > > > > >>>>>
> > > > > >>>>> 2) when you install airflow, there are two new
example DAGs
> > > > > >>>>> (last_task_only) which are going back very far
in the past
> and
> > > > > >>> scheduled
> > > > > >>>> to
> > > > > >>>>> run every hour - a bunch of dags triggered on
the first start
> > of
> > > > > >>>> scheduler
> > > > > >>>>> and hosed my CPU
> > > > > >>>>>
> > > > > >>>>> Everything else was fine and I LOVE lots of
small UI changes,
> > > which
> > > > > >>>> reduced
> > > > > >>>>> a lot my use of cli.
> > > > > >>>>>
> > > > > >>>>> Thanks again for the amazing work and an awesome
project!
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>> On Sat, Feb 11, 2017 at 9:17 AM, Jeremiah Lowin
<
> > > jlowin@apache.org
> > > > >
> > > > > >>>> wrote:
> > > > > >>>>>
> > > > > >>>>>> I was able to deploy successfully. +1 (binding)
> > > > > >>>>>>
> > > > > >>>>>> On Fri, Feb 10, 2017 at 7:37 PM Maxime Beauchemin
<
> > > > > >>>>>> maximebeauchemin@gmail.com> wrote:
> > > > > >>>>>>
> > > > > >>>>>>> +1 (binding)
> > > > > >>>>>>>
> > > > > >>>>>>> On Fri, Feb 10, 2017 at 3:44 PM, Arthur
Wiedmer <
> > > > > >>>>>> arthur.wiedmer@gmail.com>
> > > > > >>>>>>> wrote:
> > > > > >>>>>>>
> > > > > >>>>>>>> +1 (binding)
> > > > > >>>>>>>>
> > > > > >>>>>>>> On Feb 10, 2017 3:13 PM, "Dan Davydov"
<
> > > dan.davydov@airbnb.com.
> > > > > >>>>>> invalid>
> > > > > >>>>>>>> wrote:
> > > > > >>>>>>>>
> > > > > >>>>>>>>> Our staging looks good, all
the DAGs there pass.
> > > > > >>>>>>>>> +1 (binding)
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> On Fri, Feb 10, 2017 at 10:21
AM, Chris Riccomini <
> > > > > >>>>>>> criccomini@apache.org
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> wrote:
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>> Running in all environments.
Will vote after the weekend
> > to
> > > > > >>>> make
> > > > > >>>>>> sure
> > > > > >>>>>>>>>> things are working properly,
but so far so good.
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> On Fri, Feb 10, 2017 at
6:05 AM, Bolke de Bruin <
> > > > > >>>>> bdbruin@gmail.com
> > > > > >>>>>>>
> > > > > >>>>>>>>> wrote:
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>> Dear All,
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> Let’s try again!
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> I have made the THIRD
RELEASE CANDIDATE of Airflow
> 1.8.0
> > > > > >>>>>> available
> > > > > >>>>>>>> at:
> > > > > >>>>>>>>>>>
> > https://dist.apache.org/repos/dist/dev/incubator/airflow/
> > > > > >>> <
> > > > > >>>>>>>>>>>
> > https://dist.apache.org/repos/dist/dev/incubator/airflow/>
> > > > > >>> ,
> > > > > >>>>>>> public
> > > > > >>>>>>>>> keys
> > > > > >>>>>>>>>>> are available at https://dist.apache.org/repos/
> > > > > >>>>>>>> dist/release/incubator/
> > > > > >>>>>>>>>>> airflow/ <
> > > > > >>>>> https://dist.apache.org/repos/dist/release/incubator/
> > > > > >>>>>>>>> airflow/>
> > > > > >>>>>>>>>>> . It is tagged with
a local version “apache.incubating”
> > so
> > > > > >>> it
> > > > > >>>>>>> allows
> > > > > >>>>>>>>>>> upgrading from earlier
releases.
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> Two issues have been
fixed since release candidate 2:
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> * trigger_dag could
create dags with fractional
> seconds,
> > > > > >>> not
> > > > > >>>>>>>> supported
> > > > > >>>>>>>>> by
> > > > > >>>>>>>>>>> logging and UI at the
moment
> > > > > >>>>>>>>>>> * local api client trigger_dag
had hardcoded execution
> of
> > > > > >>>> None
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> Known issue:
> > > > > >>>>>>>>>>> * Airflow on kubernetes
and num_runs -1 (default) can
> > > > > >>> expose
> > > > > >>>>>> import
> > > > > >>>>>>>>>> issues.
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> I have extensively discussed
this with Alex (reporter)
> > and
> > > > > >>> we
> > > > > >>>>>>>> consider
> > > > > >>>>>>>>>>> this a known issue with
a workaround available as we
> are
> > > > > >>>> unable
> > > > > >>>>>> to
> > > > > >>>>>>>>>>> replicate this in a
different environment. UPDATING.md
> > has
> > > > > >>>> been
> > > > > >>>>>>>> updated
> > > > > >>>>>>>>>>> with the work around.
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> As these issues are
confined to a very specific area
> and
> > > > > >>> full
> > > > > >>>>>> unit
> > > > > >>>>>>>>> tests
> > > > > >>>>>>>>>>> were added I would also
like to raise a VOTE for
> > releasing
> > > > > >>>>> 1.8.0
> > > > > >>>>>>>> based
> > > > > >>>>>>>>> on
> > > > > >>>>>>>>>>> release candidate 3,
i.e. just renaming release
> > candidate 3
> > > > > >>>> to
> > > > > >>>>>>> 1.8.0
> > > > > >>>>>>>>>>> release.
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> Please respond to this
email by:
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> +1,0,-1 with *binding*
if you are a PMC member or
> > > > > >>>> *non-binding*
> > > > > >>>>>> if
> > > > > >>>>>>>> you
> > > > > >>>>>>>>>> are
> > > > > >>>>>>>>>>> not.
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> Thanks!
> > > > > >>>>>>>>>>> Bolke
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> My VOTE: +1 (binding)
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>
> > > > > >>>>>
> > > > > >>>>
> > > > > >>>
> > > > > >> --
> > > > > >> _/
> > > > > >> _/ Alex Van Boxel
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> --
>   _/
> _/ Alex Van Boxel
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message