drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From François Méthot <fmetho...@gmail.com>
Subject show files problem
Date Mon, 15 Jul 2019 18:17:44 GMT
Hi,

 We are aiming to upgrade our Drill cluster to version 1.15. On my test
environment, we found that show files operation can take close to 1+ hour
to list content of a directory with only a handful of files.

Based on Jstack, it seems a drillbit is busy collecting permission
information.
Also I see the process "ls -ld ...<path-im-querying>/subpath/files.xyz"
showing up in my ps -ef output.

I have read about the new information_schema, I suspect it is linked to
that feature, is there a flag to enable the show files behaviour from pre
1.15 versions?

The folder and files I am navigating are on an NFS mount.

Any hints or help would be appreciated

Thanks
Francois

ex:
> show files from wrk.base.`2019/`;

error... you should enable list_files_recursive
>alter system set `storage.list_files_recursive`=true

> show files from wrk.base.`2019/`;
takes hours to get result


wrk.base is  /data/
  which contain:
/2019
   /07
     /14/~1000 files
     /13/~1000 files
     /12/~1000 files
...
/2018
  ...
   /07
      /14/~1000 files
  ...

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message