spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mich Talebzadeh <mich.talebza...@gmail.com>
Subject Re: spark thrift server as hive on spark running on kubernetes, and more.
Date Fri, 10 Sep 2021 06:38:04 GMT
Hi,

Are you implying that the tool uses spark on Kubernetes as the
execution engine for Hive.

What version of Spark are you running on Kubernetes please and the
corresponding version of Hive?

Back in 2016, I did some work
<https://talebzadehmich.files.wordpress.com/2016/08/hive_on_spark_only.pdf>
with Hive running on the Spark execution engine but the progress was
restricted because of Hive and Spark library compatibility for future
releases.

So I am keen to see how this versioning issue has been rectified.

HTH,

Mich



   view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Fri, 10 Sept 2021 at 00:40, Kidong Lee <mykidong@gmail.com> wrote:

> Hi,
>
> Recently, I have open-sourced a tool called DataRoaster(
> https://github.com/cloudcheflabs/dataroaster) to provide data platforms
> running on kubernetes with ease.
> In particular, with DataRoaster, you can deploy spark thrift server on
> kubernetes easily, which is originated from my blog of
> https://itnext.io/hive-on-spark-in-kubernetes-115c8e9fa5c1.
> In addition to spark thrift server as hive on spark, there are several
> components provided by DataRoaster, for instance, hive metastore, trino,
> redash, jupyterhub, kafka.
>
> To use DataRoaster,
> - visit https://github.com/cloudcheflabs/dataroaster .
> - see also demo
> https://github.com/cloudcheflabs/dataroaster#dataroaster-demo
>
> Thank you.
>
> - Kidong Lee.
>

Mime
View raw message