airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Guziel <alex.guz...@airbnb.com.INVALID>
Subject Re: [RESULT] [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc4
Date Thu, 23 Feb 2017 00:38:36 GMT
I have some concern that this change
https://github.com/apache/incubator-airflow/pull/1939
[AIRFLOW-679] may be having issues because we are seeing lots of double triggers
of tasks and tasks being killed as a result.  





On Wed, Feb 22, 2017 4:35 PM, Dan Davydov dan.davydov@airbnb.com.INVALID  wrote:
Bumping the thread so another user can comment.




On Wed, Feb 22, 2017 at 3:12 PM, Maxime Beauchemin <

maximebeauchemin@gmail.com> wrote:




> What I meant to ask is "how much engineering effort it takes to bake a

> single RC?", I guess it depends on how much git-fu is necessary plus some

> overhead cost of doing the series of actions/commands/emails/jira.

>

> I can volunteer for 1.8.1 (hopefully I can get do it along another Airbnb

> engineer/volunteer to tag along) and will try to document/automate

> everything I can as I go through the process. The goal of 1.8.1 could be to

> basically package 1.8.0 + Dan's bugfix, and for Airbnb to get familiar with

> the process.

>

> It'd be great if you can dump your whole process on the wiki, and we'll

> improve it on this next pass.

>

> Thanks again for the mountain of work that went into packaging this

> release.

>

> Max

>

> On Wed, Feb 22, 2017 at 2:44 PM, Bolke de Bruin <bdbruin@gmail.com> wrote:

>

> > I thought you volunteered to baby sit 1.8.1 Chris ;-)?

> >

> > Sent from my iPhone

> >

> > > On 22 Feb 2017, at 23:31, Chris Riccomini <criccomini@apache.org>

> wrote:

> > >

> > > I'm +1 for doing a 1.8.1 fast follow-on

> > >

> > > On Wed, Feb 22, 2017 at 2:26 PM, Maxime Beauchemin <

> > > maximebeauchemin@gmail.com> wrote:

> > >

> > >> Our database may have edge cases that could be associated with running

> > any

> > >> previous version that may or may not have been part of an official

> > release.

> > >>

> > >> Let's see if anyone else reports the issue. If no one does, one option

> > is

> > >> to release 1.8.0 as is with a comment in the release notes, and have a

> > >> future official minor apache release 1.8.1 that would fix these minor

> > >> issues that are not deal breaker.

> > >>

> > >> @bolke, I'm curious, how long does it take you to go through one

> release

> > >> cycle? Oh, and do you have a documented step by step process for

> > releasing?

> > >> I'd like to add the Pypi part to this doc and add committers that are

> > >> interested to have rights on the project on Pypi.

> > >>

> > >> Max

> > >>

> > >>> On Wed, Feb 22, 2017 at 2:00 PM, Bolke de Bruin <bdbruin@gmail.com>

> > wrote:

> > >>>

> > >>> So it is a database integrity issue? Afaik a start_date should always

> > be

> > >>> set for a DagRun (create_dagrun) does so I didn't check the code

> > though.

> > >>>

> > >>> Sent from my iPhone

> > >>>

> > >>>> On 22 Feb 2017, at 22:19, Dan Davydov <dan.davydov@airbnb.com.

> > INVALID>

> > >>> wrote:

> > >>>>

> > >>>> Should clarify this occurs when a dagrun does not have a start
date,

> > >> not

> > >>> a

> > >>>> dag (which makes it even less likely to happen). I don't think
this

> is

> > >> a

> > >>>> blocker for releasing.

> > >>>>

> > >>>>> On Wed, Feb 22, 2017 at 1:15 PM, Dan Davydov <

> dan.davydov@airbnb.com

> > >

> > >>> wrote:

> > >>>>>

> > >>>>> I rolled this out in our prod and the webservers failed to
load due

> > to

> > >>>>> this commit:

> > >>>>>

> > >>>>> [AIRFLOW-510] Filter Paused Dags, show Last Run & Trigger
Dag

> > >>>>> 7c94d81c390881643f94d5e3d7d6fb351a445b72

> > >>>>>

> > >>>>> This fixed it:

> > >>>>> - </a> <span id="statuses_info"

> > >>>>> class="glyphicon glyphicon-info-sign" aria-hidden="true"

> title="Start

> > >>> Date:

> > >>>>> {{last_run.start_date.strftime('%Y-%m-%d %H:%M')}}"></span>

> > >>>>> + </a> <span id="statuses_info"

> > >>>>> class="glyphicon glyphicon-info-sign" aria-hidden="true"></span>

> > >>>>>

> > >>>>> This is caused by assuming that all DAGs have start dates set,
so a

> > >>> broken

> > >>>>> DAG will take down the whole UI. Not sure if we want to make
this a

> > >>> blocker

> > >>>>> for the release or not, I'm guessing for most deployments this

> would

> > >>> occur

> > >>>>> pretty rarely. I'll submit a PR to fix it soon.

> > >>>>>

> > >>>>>

> > >>>>>

> > >>>>> On Tue, Feb 21, 2017 at 9:49 AM, Chris Riccomini <

> > >> criccomini@apache.org

> > >>>>

> > >>>>> wrote:

> > >>>>>

> > >>>>>> Ack that the vote has already passed, but belated +1 (binding)

> > >>>>>>

> > >>>>>> On Tue, Feb 21, 2017 at 7:42 AM, Bolke de Bruin <

> bdbruin@gmail.com>

> > >>>>>> wrote:

> > >>>>>>

> > >>>>>>> IPMC Voting can be found here:

> > >>>>>>>

> > >>>>>>> http://mail-archives.apache.org/mod_mbox/incubator-general/

> > >>>>>> 201702.mbox/%

> > >>>>>>> 3c676BDC9F-1B55-4469-92A7-9FF309AD0EC8@gmail.com%3e
<

> > >>>>>>> http://mail-archives.apache.org/mod_mbox/incubator-general/

> > >>>>>> 201702.mbox/%

> > >>>>>>> 3C676BDC9F-1B55-4469-92A7-9FF309AD0EC8@gmail.com%3E>

> > >>>>>>>

> > >>>>>>> Kind regards,

> > >>>>>>> Bolke

> > >>>>>>>

> > >>>>>>>> On 21 Feb 2017, at 08:20, Bolke de Bruin <bdbruin@gmail.com>

> > >> wrote:

> > >>>>>>>>

> > >>>>>>>> Hello,

> > >>>>>>>>

> > >>>>>>>> Apache Airflow (incubating) 1.8.0 (based on RC4)
has been

> > accepted.

> > >>>>>>>>

> > >>>>>>>> 9 “+1” votes received:

> > >>>>>>>>

> > >>>>>>>> - Maxime Beauchemin (binding)

> > >>>>>>>> - Arthur Wiedmer (binding)

> > >>>>>>>> - Dan Davydov (binding)

> > >>>>>>>> - Jeremiah Lowin (binding)

> > >>>>>>>> - Siddharth Anand (binding)

> > >>>>>>>> - Alex van Boxel (binding)

> > >>>>>>>> - Bolke de Bruin (binding)

> > >>>>>>>>

> > >>>>>>>> - Jayesh Senjaliya (non-binding)

> > >>>>>>>> - Yi (non-binding)

> > >>>>>>>>

> > >>>>>>>> Vote thread (start):

> > >>>>>>>> http://mail-archives.apache.org/mod_mbox/incubator-

> > >>>>>>> airflow-dev/201702.mbox/%3cD360D9BE-C358-42A1-9188-

> > >>>>>>> 6C92C31A2F8B@gmail.com%3e <http://mail-archives.apache.

> > >>>>>>> org/mod_mbox/incubator-airflow-dev/201702.mbox/%3C7EB7B6D6-

> > >>>>>> 092E-48D2-AA0F-

> > >>>>>>> 15F44376A8FF@gmail.com%3E>

> > >>>>>>>>

> > >>>>>>>> Next steps:

> > >>>>>>>> 1) will start the voting process at the IPMC mailinglist.
I do

> > >> expect

> > >>>>>>> some changes to be required mostly in documentation
maybe a

> license

> > >>> here

> > >>>>>>> and there. So, we might end up with changes to stable.
As long as

> > >>> these

> > >>>>>> are

> > >>>>>>> not (significant) code changes I will not re-raise
the vote.

> > >>>>>>>> 2) Only after the positive voting on the IPMC and
finalisation I

> > >> will

> > >>>>>>> rebrand the RC to Release.

> > >>>>>>>> 3) I will upload it to the incubator release page,
then the tar

> > >> ball

> > >>>>>>> needs to propagate to the mirrors.

> > >>>>>>>> 4) Update the website (can someone volunteer please?)

> > >>>>>>>> 5) Finally, I will ask Maxime to upload it to pypi.
It seems we

> > can

> > >>>>>> keep

> > >>>>>>> the apache branding as lib cloud is doing this as well
(

> > >>>>>>> https://libcloud.apache.org/downloads.html#pypi-package
<

> > >>>>>>> https://libcloud.apache.org/downloads.html#pypi-package>).

> > >>>>>>>>

> > >>>>>>>> Jippie!

> > >>>>>>>>

> > >>>>>>>> Bolke

> > >>>>>>>

> > >>>>>>>

> > >>>>>>

> > >>>>>

> > >>>>>

> > >>>

> > >>

> >

>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message