spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bode, Meikel, NMA-CFD" <Meikel.B...@Bertelsmann.de>
Subject RE: spark thrift server as hive on spark running on kubernetes, and more.
Date Fri, 10 Sep 2021 06:24:46 GMT
Hi,

thx. Great work. Will test it 😊

Best,
Meikel Bode

From: Kidong Lee <mykidong@gmail.com>
Sent: Freitag, 10. September 2021 01:39
To: user@spark.apache.org
Subject: spark thrift server as hive on spark running on kubernetes, and more.

Hi,

Recently, I have open-sourced a tool called DataRoaster(https://github.com/cloudcheflabs/dataroaster<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcloudcheflabs%2Fdataroaster&data=04%7C01%7CMeikel.Bode%40bertelsmann.de%7C27670cbde73847d2febc08d973eb1730%7C1ca8bd943c974fc68955bad266b43f0b%7C0%7C0%7C637668275820222631%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=FC4q7xSJAqOo5ovrW%2BVWWOIobEY399c56hrQz%2BiXrdU%3D&reserved=0>)
to provide data platforms running on kubernetes with ease.
In particular, with DataRoaster, you can deploy spark thrift server on kubernetes easily,
which is originated from my blog of https://itnext.io/hive-on-spark-in-kubernetes-115c8e9fa5c1<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fitnext.io%2Fhive-on-spark-in-kubernetes-115c8e9fa5c1&data=04%7C01%7CMeikel.Bode%40bertelsmann.de%7C27670cbde73847d2febc08d973eb1730%7C1ca8bd943c974fc68955bad266b43f0b%7C0%7C0%7C637668275820222631%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=yOWYP%2B7F84bEzqjWCgviCheFx8FcoOsV4iFVYJE0LcA%3D&reserved=0>.
In addition to spark thrift server as hive on spark, there are several components provided
by DataRoaster, for instance, hive metastore, trino, redash, jupyterhub, kafka.

To use DataRoaster,
- visit https://github.com/cloudcheflabs/dataroaster<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcloudcheflabs%2Fdataroaster&data=04%7C01%7CMeikel.Bode%40bertelsmann.de%7C27670cbde73847d2febc08d973eb1730%7C1ca8bd943c974fc68955bad266b43f0b%7C0%7C0%7C637668275820232594%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2Ftlo0CC%2BMHhsfBrz9k11r5TxLwj6cwTU46Lm4RkC3Og%3D&reserved=0>
.
- see also demo https://github.com/cloudcheflabs/dataroaster#dataroaster-demo<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcloudcheflabs%2Fdataroaster%23dataroaster-demo&data=04%7C01%7CMeikel.Bode%40bertelsmann.de%7C27670cbde73847d2febc08d973eb1730%7C1ca8bd943c974fc68955bad266b43f0b%7C0%7C0%7C637668275820232594%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=EYYgLtsjHTdN4f8S2D1E%2F5mjX9sYaNaI2gp4xQN9PDU%3D&reserved=0>

Thank you.

- Kidong Lee.
Mime
View raw message