airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Huang <dxhu...@gmail.com>
Subject Pulling XCom from parent DAG in a sub DAG template
Date Thu, 02 Feb 2017 00:23:37 GMT
Hey everyone,
I currently have a parent DAG that generates a filename based on the
current timestamp. That same filename needs to be used down the line in
multiple operators, including some subdag operators. So I was hoping to use
XComs to pass this filename along. It works fine for operators within the
parent DAG, but not for any operators in the sub DAGs because the dag_id
arg on xcom_pull() defaults to self.dag_id. I'm pulling this XCom value
from templates, so I don't want to have to hardcode a parent's dag id in it
because it should work for different parent dags. The sub dag template
would still have to make assumptions about what is available in xcom under
a task id, but I think that's alright?

So my question is if there is a way to programmatically specify the parent
dag id in an xcom pull from within a template or if there's a better way of
doing what I'm trying to achieve. Note, I also have sub dags within sub
dags that require the same filename.

If there is no way to this already, I did come across
https://issues.apache.org/jira/browse/AIRFLOW-54 and wondered if we need
something similar for the dag_id arg, like ti.xcom_pull(dag_id='@parent',
task_ids='foo').

Thanks,
Daniel

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message