drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Tzvetanov Grigorov <mgrigo...@apache.org>
Subject Test Apache Drill on Linux ARM64
Date Fri, 07 May 2021 12:02:11 GMT
Hello Drill developers,

Recently I've tried to build Apache Drill on ARM64 hardware running on Linux.
I have found few issues which are described in issue https://issues.apache.org/jira/browse/DRILL-7911

I've created few Pull Requests with fixes for each issue:
- https://github.com/apache/drill/pull/2217 - use TestContainers-MySQL instead of Wix-Embedded-MySQL
- https://github.com/apache/drill/pull/2218 - Disable Storage-Splunk unit tests on Linux ARM64
because there is no Docker image of Splunk for Linux/arm64
- https://github.com/apache/drill/pull/2219 - Increase Max Direct Memory

Now I would like to suggest adding CI testing on ARM64 to prevent regressions in the future.
The problem is that GitHub Actions (the CI system used by Apache Drill) does not yet support
ARM64 architecture.

Here are the possible solutions I am aware of:

* use TravisCI only for running `mvn install` on Linux ARM64
- TravisCI supports Linux ARM64 out of the box and the config is quite simple
- Might be useful later if someone wants to add testing on Linux s390x
- Use a second CI for such specific purpose

* Use GitHub Actions to run the build at a remote Kubernetes cluster with ARM64 nodes
More details about this approach could be read at https://martin-grigorov.medium.com/githubactions-build-and-test-on-huaweicloud-arm64-af9d5c97b766
Disclaimer: I work for OpenLab Testing and Huawei sponsor us, so I can get you a free account
at HuaweiCloud for such setup. 
The same setup could be used with any other Kubernetes provider!

* Use CircleCI instead of GitHub Actions
- native support for both x86_64 and aarch64  (https://github.com/CircleCI-Public/arm-preview-docs)
- CircleCI allows connecting via SSH to a builder node. This way one can debug issues
- higher job timeout (5h) - https://circleci.com/docs/2.0/runner-installation/#runner-max_run_time.
Currently Github Actions often fail due to build timeouts of 90mins
- it is less crowded than the Apache organization at GitHub Actions (https://ibb.co/RpFyQQy),
so there is less wait time for the build
- work is required to migrate from GitHub Actions to CircleCI

I volunteer to do the work for any of these options. Just please let me know which one is
your preferred one!


View raw message