spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From qingyang li <liqingyang1...@gmail.com>
Subject on shark, is tachyon less efficient than memory_only cache strategy ?
Date Tue, 08 Jul 2014 08:18:51 GMT
hi, when i create a table, i can point the cache strategy using shark.cache,
i think "shark.cache=memory_only"  means data are managed by spark, and
data are in the same jvm with excutor;   while  "shark.cache=tachyon"
 means  data are managed by tachyon which is off heap, and data are not in
the same jvm with excutor,  so spark will load data from tachyon for each
query sql , so,  is  tachyon less efficient than memory_only cache strategy
 ?
if yes, can we let spark load all data once from tachyon  for all sql query
 if i want to use tachyon cache strategy since tachyon is more HA than
memory_only ?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message