spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Soumya Simanta <soumya.sima...@gmail.com>
Subject Re: Spark as Relational Database
Date Sun, 26 Oct 2014 03:34:35 GMT
1. What data store do you want to store your data in ? HDFS, HBase,
Cassandra, S3 or something else?
2. Have you looked at SparkSQL (https://spark.apache.org/sql/)?

One option is to process the data in Spark and then store it in the
relational database of your choice.




On Sat, Oct 25, 2014 at 11:18 PM, Peter Wolf <opus111@gmail.com> wrote:

> Hello all,
>
> We are considering Spark for our organization.  It is obviously a superb
> platform for processing massive amounts of data... how about retrieving it?
>
> We are currently storing our data in a relational database in a star
> schema.  Retrieving our data requires doing many complicated joins across
> many tables.
>
> Can we use Spark as a relational database?  Or, if not, can we put Spark
> on top of a relational database?
>
> Note that we don't care about SQL.  Accessing our data via standard
> queries is nice, but we are equally happy (or more happy) to write Scala
> code.
>
> What is important to us is doing relational queries on huge amounts of
> data.  Is Spark good at this?
>
> Thank you very much in advance
> Peter
>

Mime
View raw message