kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anto Aravinth <anto.aravinth....@gmail.com>
Subject Re: How to resolve Kafka within docker environment?
Date Mon, 08 Jun 2020 11:53:29 GMT
Hello Ricardo,

Thanks much for your help. In deed it have fixed the problem.

On Thu, Jun 4, 2020 at 8:31 PM Ricardo Ferreira <riferrei@riferrei.com>
wrote:

> Anto,
>
> I am not 100% familiar with this image `confluentinc/cp-kafka` but there
> is a few things that you should try:
>
> 1) Make sure your `kafka` containers has this name set appropriately
>
> ```
>
>   kafka:
>     image: confluentinc/cp-enterprise-kafka:5.5.0
>     container_name: *kafka*
>     depends_on:
>       - zookeeper
>
> ```
>
> 2) Set the list of protocols that Kafka will communicate externally
>
> ```
>
> KAFKA_LISTENER_SECURITY_PROTOCOL_MAP:
> PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
>
> ```
>
> 3) In your app container; provide the bootstrap server value with double
> quotes
>
> ```
>
>             depends_on:
>               - kafka
>               - zookeeper
>               - postgres
>               - connector
>             environment:
>               BOOTSTRAP_SERVERS: *"kafka:9092"*
>
> ```
>
> Thanks,
>
> -- riferrei
> On 6/3/20 9:55 PM, Anto Aravinth wrote:
>
> I have a spring application, which should connect to the kafka. This is
> what my Docker file looks for spring application:
>
> ```
> FROM maven:3-jdk-11 as builder
> # create app folder for sources
> RUN mkdir -p /build
> WORKDIR /build
> COPY pom.xml /build
> #Download all required dependencies into one layer
> RUN mvn -B dependency:resolve dependency:resolve-plugins
> #Copy source code
> COPY src /build/src
> # Build application
> RUN mvn package
>
> FROM openjdk:8-jdk-alpine
> RUN addgroup -S spring && adduser -S spring -G spring
> USER spring:spring
> ARG JAR_FILE=target/*.jar
> COPY ${JAR_FILE} app.jar
> ENTRYPOINT ["java","-jar","/app.jar"]
> ```
>
> The kafka/Postgres/zookeeper everything other, comes from Docker images. So
> thought will run the application in docker compose, so that it looks like
> the following:
>
>     version: '3.1'
>     services:
>         postgres:
>             image: debezium/postgres
>             environment:
>               POSTGRES_PASSWORD: qwerty
>               POSTGRES_USER: appuser
>             volumes:
>                - ./postgres:/data/postgres
>             ports:
>               - 6532:6532
>         zookeeper:
>             image: confluentinc/cp-zookeeper
>             ports:
>               - "2181:2181"
>             environment:
>               ZOOKEEPER_CLIENT_PORT: 2181
>         kafka:
>             image: confluentinc/cp-kafka
>             depends_on:
>               - zookeeper
>               - postgres
>             ports:
>               - "9092:9092"
>             environment:
>               KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
>               KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
>               KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
>               KAFKA_LOG_CLEANER_DELETE_RETENTION_MS: 5000
>               KAFKA_BROKER_ID: 1
>               KAFKA_MIN_INSYNC_REPLICAS: 1
>               KAFKA_ADVERTISED_HOST_NAME: kafka
>         connector:
>             image: debezium/connect:latest
>             ports:
>               - "8083:8083"
>             environment:
>               GROUP_ID: 1
>               CONFIG_STORAGE_TOPIC: my_connect_configs
>               OFFSET_STORAGE_TOPIC: my_connect_offsets
>               BOOTSTRAP_SERVERS: kafka:9092
>             depends_on:
>               - zookeeper
>               - postgres
>               - kafka
>         app-server:
>         # Configuration for building the docker image for the backend
> service
>             build:
>               context: . # Use an image built from the specified dockerfile
> in the `polling-app-server` directory.
>               dockerfile: Dockerfile
>             ports:
>               - "8080:8080" # Forward the exposed port 8080 on the
> container to port 8080 on the host machine
>             restart: always
>             depends_on:
>               - kafka
>               - zookeeper
>               - postgres
>               - connector
>             environment:
>               BOOTSTRAP_SERVERS: kafka:9092
>
> I pass an environment variable `BOOTSTRAP_SERVERS` with value as
> `kafka:9092` (since `localhost:9092` doesn't work inside my docker
> environment).
>
> And in my spring code, I get the value like the following:
>
> ```
> System.getenv("bootstrap.servers")
> // or
> System.getenv("BOOTSTRAP_SERVERS")
> ```
>
> however, when I run `docker-compose up`, I get the value as `null` for the
> above Java code. Not sure, what is the best way to get the docker resolved
> network for my kafka.
>
>  So how to fix it so that java code picks up the Kafka broker inside the
> docker environment?
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message