spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sea aj <saj3...@gmail.com>
Subject Training A ML Model on a Huge Dataframe
Date Wed, 23 Aug 2017 12:27:57 GMT
Hi,

I am trying to feed a huge dataframe to a ml algorithm in Spark but it
crashes due to the shortage of memory.

Is there a way to train the model on a subset of the data in multiple steps?

Thanks



<https://mailtrack.io/> Sent with Mailtrack
<https://mailtrack.io/install?source=signature&lang=en&referral=saj3saj@gmail.com&idSignature=22>

Mime
View raw message