spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yu Wei <yu20...@hotmail.com>
Subject Re: Is it good choice to use DAO to store results generated by spark application?
Date Wed, 20 Jul 2016 06:33:50 GMT
Hi Ted,

I also noticed HBASE-13992.

I never used stuff similar as DAO.

As a general rule, which is better choice when working with spark, hbase? hbase-spark module,
DAO or hbase client api?


I'm beginner to big data.

Any guidance is very helpful for me.


Thanks,

Jared

________________________________
From: Ted Yu <yuzhihong@gmail.com>
Sent: Wednesday, July 20, 2016 12:14:17 PM
To: Andrew Ehrlich
Cc: Yu Wei; user@spark.apache.org
Subject: Re: Is it good choice to use DAO to store results generated by spark application?

hbase-spark module is in the up-coming hbase 2.0 release.
Currently it is in master branch of hbase git repo.

FYI

On Tue, Jul 19, 2016 at 8:27 PM, Andrew Ehrlich <andrew@aehrlich.com<mailto:andrew@aehrlich.com>>
wrote:
There is a Spark<->HBase library that does this.  I used it once in a prototype (never
tried in production through): http://blog.cloudera.com/blog/2015/08/apache-spark-comes-to-apache-hbase-with-hbase-spark-module/

On Jul 19, 2016, at 9:34 AM, Yu Wei <yu2003w@hotmail.com<mailto:yu2003w@hotmail.com>>
wrote:

Hi guys,

I write spark application and want to store results generated by spark application to hbase.
Do I need to access hbase via java api directly?
Or is it better choice to use DAO similar as traditional RDBMS?  I suspect that there is major
performance downgrade and other negative impacts using DAO. However, I have little knowledge
in this field.

Any advice?

Thanks,
Jared



Mime
View raw message