spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastian Piu <sebastian....@gmail.com>
Subject Re: Looking for a developer to help us with a small ETL project using Spark and Kubernetes
Date Fri, 19 Jul 2019 06:27:34 GMT
Hey Warren,

I've done similar integrations in the past, are you looking for a freelance
dev to achieve this? I'm based in the UK.

Cheers
Seb


On Thu, 18 Jul 2019, 11:47 pm Information Technologies, <
it@digitalearthnetwork.com> wrote:

> Hello,
>
> We are looking for a developer to help us with a small ETL project using
> Spark and Kubernetes. Here are some of the requirements:
>
> 1. We need a REST API to run and schedule jobs. We would prefer this done
> in Node.js but can be done using Java. The REST API will not be available
> to the public.
> 2. We need an easy way to create new jobs in Java without deploying the
> whole server again.
> 3. We want jobs deployed/ran using Kubernetes.
> 4. Must be able to scale to 1000s of ETL jobs.
> 5. Source for data will be one REST API.
> 6. Destination for data will be one Couchbase Database cluster. (Couchbase
> also uses a REST API)
> 7. I am not sure how many records will be processed per job.
> 8. The data is mostly sales related data.
>
> I know there are commercial ETL solutions that do everything I want. We
> are looking for something simple and do not need a fancy UI to describe our
> ETL. We want to use Spark and Java to programmatically describe out ETL
> jobs.
>
> Please let me know if you are interested.
>
> Thanks,
>
> Warren Bell
> --
>
>
> **********************************************************************
>
> This
>  email and any files transmitted with it are confidential and intended
>
> solely for the use of the individual or entity to whom they are
>
> addressed. They may not be disseminated or distributed to persons or
>
> entities other than the ones intended without the authority of the
> sender.
> If you have received this email in error or are not the
>  intended
> recipient, you may not use, copy, disseminate or distribute
> it. Delete it
> immediately from your system and notify the sender
> promptly
>  by email that
> you have done so. This footnote also confirms that this
> email message has
> been scanned for the presence of computer viruses.
>
>
> **********************************************************************
>
>
> Please consider the environment before printing
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>

Mime
View raw message