drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From François Méthot <fmetho...@gmail.com>
Subject Re: show files problem
Date Tue, 16 Jul 2019 13:28:01 GMT
Good catch!
Indeed I was using a SNAPSHOT version.
Since I have no choice to upgrade now, I will take the opportunity to get
the latest version of drill!

Thanks Arina,

On Mon, 15 Jul 2019 at 15:22, Arina Yelchiyeva <arina.yelchiyeva@gmail.com>
wrote:

> Hi,
>
> The issue you are describing was fixed in DRILL-6753 [1] which was
> included into 1.15.0 release.
> I suspect you are trying to upgrade to some 1.15.0-SNAPSHOT version rather
> than on 1.15.0 final version.
>
> [1] https://issues.apache.org/jira/browse/DRILL-6753
>
> Kind regards,
> Arina
>
> > On Jul 15, 2019, at 9:17 PM, François Méthot <fmethot78@gmail.com>
> wrote:
> >
> > Hi,
> >
> > We are aiming to upgrade our Drill cluster to version 1.15. On my test
> > environment, we found that show files operation can take close to 1+ hour
> > to list content of a directory with only a handful of files.
> >
> > Based on Jstack, it seems a drillbit is busy collecting permission
> > information.
> > Also I see the process "ls -ld ...<path-im-querying>/subpath/files.xyz"
> > showing up in my ps -ef output.
> >
> > I have read about the new information_schema, I suspect it is linked to
> > that feature, is there a flag to enable the show files behaviour from pre
> > 1.15 versions?
> >
> > The folder and files I am navigating are on an NFS mount.
> >
> > Any hints or help would be appreciated
> >
> > Thanks
> > Francois
> >
> > ex:
> >> show files from wrk.base.`2019/`;
> >
> > error... you should enable list_files_recursive
> >> alter system set `storage.list_files_recursive`=true
> >
> >> show files from wrk.base.`2019/`;
> > takes hours to get result
> >
> >
> > wrk.base is  /data/
> >  which contain:
> > /2019
> >   /07
> >     /14/~1000 files
> >     /13/~1000 files
> >     /12/~1000 files
> > ...
> > /2018
> >  ...
> >   /07
> >      /14/~1000 files
> >  ...
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message