spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Masf <masfwo...@gmail.com>
Subject SQLContext load. Filtering files
Date Wed, 19 Aug 2015 17:16:17 GMT
Hi.

I'd like to read Avro files using this library
https://github.com/databricks/spark-avro

I need to load several files from a folder, not all files. Is there some
functionality to filter the files to load?

And... Is is possible to know the name of the files loaded from a folder?

My problem is that I have a folder where an external process is inserting
files every X minutes and I need process these files once, and I can't
move, rename or copy the source files.


Thanks
-- 

Regards
Miguel Ángel

Mime
View raw message