spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christophe Préaud <>
Subject Re: DataFrame support for hadoop glob patterns
Date Wed, 09 Mar 2016 16:14:42 GMT

Unless I've misunderstood what you want to achieve, you could use:"/mnt/views-p/base/2016/01/*/*-xyz.json"))


On 09/03/16 15:24, Ted Yu wrote:
Hadoop glob pattern doesn't support multi level wildcard.


On Mar 9, 2016, at 6:15 AM, Koert Kuipers <<><>>

if its based on HadoopFsRelation shouldn't it support it? HadoopFsRelation handles globs

On Wed, Mar 9, 2016 at 8:56 AM, Ted Yu <<>>
This is currently not supported.

On Mar 9, 2016, at 4:38 AM, Jakub Liska <<><>>


is something like this possible?"/mnt/views-p/base/2016/01/*/*-xyz.json")

I switched to DataFrames because my source files changed from TSV to JSON
but now I'm not able to load the files as I did before. I get this error if I try that :

Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention exclusive
de leurs destinataires. Si vous n'êtes pas le destinataire de ce message, merci de le détruire
et d'en avertir l'expéditeur.

View raw message