spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pankaj arora <pankajarora.n...@gmail.com>
Subject Re: Use Case of mutable RDD - any ideas around will help.
Date Fri, 12 Sep 2014 18:31:02 GMT
I think i should elaborate usecase little more. 

So we have UI dashboard whose response time is quite fast as all the data is
cached. Users query data based on time range and also there is always new
data coming into the system at predefined frequency lets say 1 hour. 

As you said i can uncache tables it will basically drop all data from
memory. 
I cannot afford losing my cache even for short interval. As all queries from
UI will get slow till the time cache loads again. UI response time needs to
be predictable and shoudl be fast enough so that user does not get
irritated. 

Also i cannot keep two copies of data(till newrdd materialize) into memory
as it will surpass total available memory in system. 



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Re-Use-Case-of-mutable-RDD-any-ideas-around-will-help-tp14095p14112.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message