spot-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tadd Wood <taddw...@apache.org>
Subject [SPOT-266] ml_ops.sh script should use Spark 2?
Date Fri, 20 Mar 2020 21:23:57 GMT
I notice there’s a dependency on Spark 2.1+ that didn’t get resolved per
this JIRA (https://issues.apache.org/jira/projects/SPOT/issues/SPOT-266). I
propose adding a check in this script to verify the user’s version of Spark
and if it's not at least 2.1 the script will exit and indicate that the
user should upgrade their version of Spark. We also need to replace
spark-submit with spark-submit2. Any thoughts for/against updating this
script?

Also would anyone be willing to update the documentation on the Spot
website to reflect the correct required version of Spark to run Spot?

Thank you,
Tadd Wood

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message