parquet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wes McKinney <wesmck...@gmail.com>
Subject Re: Passing File Descriptors in the Low-Level API
Date Thu, 14 Mar 2019 19:42:51 GMT
hi Brian,

This is mostly an Arrow platform question so I'm copying the Arrow mailing list.

You can open a file using an existing file descriptor using ReadableFile::Open

https://github.com/apache/arrow/blob/master/cpp/src/arrow/io/file.h#L145

The documentation for this function says:

"The file descriptor becomes owned by the ReadableFile, and will be
closed on Close() or destruction."

If you want to do the equivalent thing, but using memory mapping, I
think you'll need to add a corresponding API to MemoryMappedFile. This
is more perilous because of the API requirements of mmap -- you need
to pass the right flags and they may need to be the same flags that
were passed when opening the file descriptor, see

https://github.com/apache/arrow/blob/master/cpp/src/arrow/io/file.cc#L378

and

https://github.com/apache/arrow/blob/master/cpp/src/arrow/io/file.cc#L476

- Wes

On Thu, Mar 14, 2019 at 1:47 PM Brian Bowman <Brian.Bowman@sas.com> wrote:
>
>  The ReadableFile class (arrow/io/file.cc) has utility methods where a FileDescriptor
is either passed in or returned, but I don’t see how this surfaces through the API.
>
> Is there a way for application code to control the open lifetime of mmap()’d Parquet
files by passing an already open FileDescriptor to Parquet low-level API open/close methods?
>
> Thanks,
>
> Brian
>

Mime
View raw message