Use a format that has built-in indexes, such as Parquet or Orc. Do not forget to sort the data on the columns that your filter on.

On 14 Aug 2016, at 05:03, Taotao.Li <> wrote:

hi, guys, does Spark SQL support indexes?  if so, how can I create an index on my temp table? if not, how can I handle some specific queries on a very large table? it would iterate all the table even though all I want is just a small piece of that table.

great thanks, 

Quant | Engineer | Boy