samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jagadish Venkatraman <>
Subject Re: Running Apache Samza on Kubernetes cluster with Zookeeper
Date Tue, 30 Apr 2019 17:15:01 GMT
+Weiqing Yang, who gave a community talk on this at KubeCon this year.

Hi Stefano,

Running Samza-standalone + Zk on Kubernetes should be no different than
running any other application on Kubernetes.

At a high-level, you would:

   1. Package your application as a container image - just as you would for
   any other application.
   2. Deploy it using Kubectl

>From this post <> sharing how to
run Samza + Zk on K8s (It is prior to Samza 1.1 - please skip the section
on low-level jobs)

*"By comparison, using the high-level StreamApplication API is considerably
simpler. There's actually very little to it: just package the application
in a way that can be executed and deploy it using StatefulSet directly,
instead of via the KubernetesJob.*

*Running gradle dockerDistTar will produce a
folder example/build/docker which contains a Dockerfile and everything
necessary to run the example application in Kubernetes. Running gradle
dockerBuildImage will install said image locally, at which point you can
run kubectl create -f example/k8s/app.yaml*

*If you're planning to use the KV store with a volume mount, keep in mind
that Samza hard-codes the location to {user.dir}/state. The combination of
the generated Docker image and app.yaml mount point works because the image
runs the application from / and the mount point is /state."*
A related (a bit-outdated) email thread:

On Tue, Apr 30, 2019 at 8:46 AM Stefano Rebora <> wrote:

> Hello,
> I’m trying to develop Apache Samza on a Kubernetes cluster using Zookeeper
> for coordination.
> Is there a suggested or working example to follow?
> Thank you in advance,
> Stefano

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message