spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Magnus Nilsson <ma...@kth.se>
Subject Re: Schema store for Parquet
Date Wed, 04 Mar 2020 19:09:53 GMT
Apache Atlas is the apache data catalog. Maybe want to look into that. It
depends on what your use case is.

On Wed, Mar 4, 2020 at 8:01 PM Ruijing Li <liruijing09@gmail.com> wrote:

> Thanks Lucas and Magnus,
>
> Would there be any open source solutions other than Apache Hive metastore,
> if we don’t wish to use Apache Hive and spark?
>
> Thanks.
>
> On Wed, Mar 4, 2020 at 10:40 AM lucas.gary@gmail.com <lucas.gary@gmail.com>
> wrote:
>
>> Or AWS glue catalog if you're in AWS
>>
>> On Wed, 4 Mar 2020 at 10:35, Magnus Nilsson <magnn@kth.se> wrote:
>>
>>> Google hive metastore.
>>>
>>> On Wed, Mar 4, 2020 at 7:29 PM Ruijing Li <liruijing09@gmail.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Has anyone explored efforts to have a centralized storage of schemas of
>>>> different parquet files? I know there is schema management for Avro, but
>>>> couldn’t find solutions for parquet schema management. Thanks!
>>>> --
>>>> Cheers,
>>>> Ruijing Li
>>>>
>>> --
> Cheers,
> Ruijing Li
>

Mime
View raw message