airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sid Anand <>
Subject Re: Use Docker for running Airflow tests
Date Thu, 06 Sep 2018 01:45:40 GMT
Any chance you can document a flow that ties the CI build artifacts to this
repo and how we would use them? Would our build times go down?


On Sun, Aug 19, 2018 at 11:43 AM Driesprong, Fokko <>

> Hi all,
> Gerardo is doing some awesome work on Dockerizing the CI pipeline. The PR
> is still open here:
> To make this work, and not store any Docker images in private repositories,
> I've created a new repository which stores the Docker image which contains
> a lot of the dependencies we use to test against. The repository is here:
> For most of the tests, we use mocks, but sometimes we also use the actual
> service, for example for some Hadoop, Kerberos, Hive functionality. The
> images are build in the Dockerhub service:
> I've created it for now. I'm open for another name, but Airflow was already
> taken, and hyphens are not allowed.
> This change will greatly decouple our dependency on Travis and will make it
> much easier to test the code locally. Instead of setting up tox
> environments, we can just use Docker instead. I'm really enthusiastic about
> the change, and really thankful for Gerardo to put all this effort in
> there.
> Recently Holden Karau did a lifestream
> <>, and did her first PR. I
> think the most important learning from it was that testing was not as
> trivial as it should be. Therefore I think we should see if we can merge
> the PR from Gerardo soon.
> Cheers, Fokko

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message