arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe L. Korn" <uw...@xhochy.com>
Subject Re: CMake refactor Heads-up
Date Sat, 16 Mar 2019 15:31:32 GMT
Hello,

> Am 16.03.2019 um 14:30 schrieb Suvayu Ali <fatkasuvayu+linux@gmail.com>:
> 
> Hello Uwe,
> 
>> On Fri, Mar 15, 2019 at 10:38:32AM -0400, Uwe L. Korn wrote:
>> 
>> we have merged the CMake refactor yesterday https://github.com/apache/arrow/pull/3688
and this means that the build system behaves a bit different. The main differences are:
>> 
> 
> A few more comments:
> 
> 1. There was an extra ARROW_GANDIVA line in the docker build script.  So I submitted
a PR: https://github.com/apache/arrow/pull/3937.
> 
> 2. I don't know if this is intentional, but jemalloc and rapidjson aren't detected on
my system.
> 
>   # rpm -qa rapidjson\* jemalloc\*
>   rapidjson-devel-1.1.0-6.fc28.noarch
>   jemalloc-devel-5.0.1-5.fc28.x86_64
>   jemalloc-5.0.1-5.fc28.x86_64
> 
>   # from cmake output
>   -- BOOST_VERSION: 1.67.0
>   -- BROTLI_VERSION: v0.6.0
>   -- BZIP2_VERSION: 1.0.6
>   -- CARES_VERSION: 1.15.0
>   -- DOUBLE_CONVERSION_VERSION: v3.1.1
>   -- FLATBUFFERS_VERSION: v1.10.0
>   -- GBENCHMARK_VERSION: v1.4.1
>   -- GFLAGS_VERSION: v2.2.0
>   -- GLOG_VERSION: v0.3.5
>   -- GRPC_VERSION: v1.18.0
>   -- GTEST_VERSION: 1.8.1
>   -- JEMALLOC_VERSION: 17c897976c60b0e6e4f4a365c751027244dada7a
>   -- LZ4_VERSION: v1.8.3
>   -- ORC_VERSION: 1.5.4
>   -- PROTOBUF_VERSION: v3.6.1
>   -- RAPIDJSON_VERSION: 2bbd33b33217ff4a73434ebf10cdac41e2ef5e34
>   -- RE2_VERSION: 2018-10-01
>   -- SNAPPY_VERSION: 1.1.3
>   -- THRIFT_VERSION: 0.12.0
>   -- ZLIB_VERSION: 1.2.8
>   -- ZSTD_VERSION: v1.3.7

Not sure if it is detected or not as this is missing from the log above. That log lists only
the bundled versions that would be installed if the package is not found on your system. Whether
or not system RapidJSON is used will be logged some lines later. 

For jemalloc the situation is slightly different. We have vendored upstream's head of the
4.x branch as that was until some time ago the only version that worked reliably on all systems
together with Arrow. We could have a look again at using the newest jemalloc version again.

> 
> 3. There does not seem to be an uninstall target.

CMake doesn't provide this and their FAQ https://gitlab.kitware.com/cmake/community/wikis/FAQ#can-i-do-make-uninstall-with-cmake
sounds reasonable to me. I guess package managers are better at removing Arrow again than
our CMake scripts. Especially as we would only be aware of the currently build files and not
the ones of previous versions. For example current master builds libarrow.so.13 but is not
aware of the libarrow.so.12 of previous releases.

> 4. AFAIU, the pyarrow build expects the libraries in $CMAKE_INSTALL_PREFIX/lib.  This
will never be accepted by a distro.  I do realise this one is probably hard to resolve, given
how the builds are setup at the moment.

This should be resolvable. One of the odeas of the recent refractor was to get rid of so hard
assumptions. We need to carry this on also on the Python side. Can you open a JIRA with a
link to the code that makes this assumption?

Uwe 

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message