drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastian Fischmeister <sfisc...@uwaterloo.ca>
Subject Re: Querying multiple s3 buckets
Date Mon, 23 Nov 2020 18:08:48 GMT
Since s3n://test*/ doesn't seem to be easily feasible, how do you combine a query on multiple
s3 buckets into a single one?

For example, I essentially want to execute 'find / -name "test*_r*.parquet"' on a set of S3
buckets:

select fqn as f from ??? where regexp_matches(f,".*test.*_r.*.parquet");

Regards,
  Sebastian

Nitin Pawar <nitinpawar432@gmail.com> writes:

> you can create multiple storages with different names to each bucker and
> query them in a single query
>
> On Sun, Nov 22, 2020 at 10:38 PM Sebastian Fischmeister <
> sfischme@uwaterloo.ca> wrote:
>
>> Hi,
>>
>> I set up drill with min.io and have multiple buckets in minio. I was able
>> to setup an s3 connection to a single bucket, however, I actually want to
>> run a query on all files in all buckets. Is this possible?
>>
>> Regards,
>>   Sebastian
>>
>
>
> -- 
> Nitin Pawar

Mime
View raw message