mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pat Ferrel <>
Subject Re: spark-itemsimilarity out of memory problem
Date Mon, 22 Dec 2014 17:30:42 GMT
The job has an option -sem to set the spark.executor.memory config. Also you can change runtime
job config with -D:key=value to access any of the Spark config values.

On Dec 21, 2014, at 11:44 PM, AlShater, Hani <> wrote:

Hi All,

I am trying to use spark-itemsimilarity on 160M user interactions dataset.
The job launches and running successfully for small data 1M action.
However, when trying for the larger dataset, some spark stages continuously
fail with out of memory exception.

I tried to change the from spark default
configuration, but I face the same issue again. How could I configure spark
when using spark-itemsimilarity, or how to overcome this out of memory

Can you please advice ?


Hani Al-Shater | Data Science Manager - <>
Mob: +962 790471101 | Phone: +962 65821236 | Skype: | <> |
Nouh Al Romi Street, Building number 8, Amman, Jordan


*Download free <> mobile apps for iPhone 
<>, iPad 
<>, Android 
<> or Windows 
**and never 
miss a deal! *

View raw message