spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean R. Owen (Jira)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-29553) This problemis about using native BLAS to improvement ML/MLLIB performance
Date Sat, 26 Oct 2019 22:33:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-29553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sean R. Owen updated SPARK-29553:
---------------------------------
    Priority: Minor  (was: Major)

How did you try to set it? it sounds like you found one way to set it correctly. You can also
modify spark-env.sh, but I'm not clear that's what you did

> This problemis about using native BLAS to improvement ML/MLLIB performance
> --------------------------------------------------------------------------
>
>                 Key: SPARK-29553
>                 URL: https://issues.apache.org/jira/browse/SPARK-29553
>             Project: Spark
>          Issue Type: Improvement
>          Components: ML, MLlib
>    Affects Versions: 2.3.0, 2.4.4
>            Reporter: WuZeyi
>            Priority: Minor
>              Labels: performance
>
> I use {color:#ff0000}native BLAS{color} to improvement ML/MLLIB performance on Yarn.
> The file {color:#ff0000}spark-env.sh{color} which is modified by SPARK-21305 said that
I should set {color:#ff0000}OPENBLAS_NUM_THREADS=1{color} to disable multi-threading of
OpenBLAS, but it does not take effect.
> I modify {color:#ff0000}spark.conf{color} to set  {color:#FF0000}spark.executorEnv.OPENBLAS_NUM_THREADS=1{color},and
the performance improve.
>   
>   
>  I think MKL_NUM_THREADS is the same.
>   



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message