spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Darshan Singh <>
Subject Logical Plan
Date Thu, 30 Jun 2016 20:58:02 GMT
Is there a way I can use same Logical plan for a query. Everything will be
same except underlying file will be different.

Issue is that my query has around 700 columns and Generating logical plan
takes 20 seconds and it happens every 2 minutes but every time underlying
file is different.

I do not know these files in advance so I cant create the table on
directory level. These files are created and then used in the final query.


View raw message