spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabor Somogyi <gabor.g.somo...@gmail.com>
Subject Re: how to get spark-sql lineage
Date Thu, 16 May 2019 12:38:03 GMT
Hi,

spark.lineage.enabled is Cloudera specific and doesn't work with vanilla
Spark.

BR,
G


On Thu, May 16, 2019 at 4:44 AM lk_spark <lk_spark@163.com> wrote:

> hi,all:
>         When I use spark , if I run some SQL to do ETL how can I get
> lineage info. I found that , CDH spark have some config about lineage :
>         spark.lineage.enabled=true
>         spark.lineage.log.dir=/var/log/spark2/lineage
>         Are they also work for apache spark ?
>
> 2019-05-16
> ------------------------------
> lk_spark
>

Mime
View raw message