spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rong Ou <rong...@gmail.com>
Subject building docker images for GPU
Date Fri, 08 Feb 2019 22:27:49 GMT
Hi spark dev,

I created a JIRA issue a while ago (
https://issues.apache.org/jira/browse/SPARK-26398) to add GPU support to
Spark docker images, and sent a PR (
https://github.com/apache/spark/pull/23347) that went through several
iterations. It was suggested that it should be discussed on the dev mailing
list, so here we are. Please chime in if you have any questions or concerns.

A little more background. I mainly looked at running XGBoost on Spark using
GPUs. Preliminary results have shown that there is potential for
significant speedup in training time. This seems like a popular use case
for Spark. In any event, it'd be nice for Spark to have better support for
GPUs. Building gpu-enabled docker images seems like a useful first step.

Thanks,

Rong

Mime
View raw message