airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flo Rance <troura...@gmail.com>
Subject PostgreSQL hook
Date Wed, 20 Mar 2019 15:30:39 GMT
Hi,

I don't know if it's the correct place to ask for that.

I'm trying to implement one of my cronjob using airflow. One of the tasks
is to load files in a temporary table and then update another table in a
postgres db.
For that, I was previously using a sql script like that:

BEGIN;

CREATE TEMP TABLE catalog_tmp ON COMMIT DROP AS SELECT * FROM catalog WITH
NO DATA;

\COPY catalog_tmp (...) FROM '/home/cat/catalog.csv' WITH DELIMITER ';' CSV
ENCODING 'LATIN1' NULL '';

DELETE FROM catalog_tmp WHERE code IS NULL;
...
COMMIT;

I would like to replace \copy with the copy_expert from postgresql hook. Is
that realistic ?
If yes, how can I combine a sql script and that hook in one task ?

Regards,
Flo

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message