kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ricardo Ferreira <rifer...@riferrei.com>
Subject Re: How to resolve Kafka within docker environment?
Date Thu, 04 Jun 2020 15:01:48 GMT
Anto,

I am not 100% familiar with this image `confluentinc/cp-kafka` but there 
is a few things that you should try:

1) Make sure your `kafka` containers has this name set appropriately

```

   kafka:
     image: confluentinc/cp-enterprise-kafka:5.5.0
     container_name: *kafka*
     depends_on:
       - zookeeper

```

2) Set the list of protocols that Kafka will communicate externally

```

KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 
PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT

```

3) In your app container; provide the bootstrap server value with double 
quotes

```

             depends_on:
               - kafka
               - zookeeper
               - postgres
               - connector
             environment:
               BOOTSTRAP_SERVERS: *"kafka:9092"*

```

Thanks,

-- riferrei

On 6/3/20 9:55 PM, Anto Aravinth wrote:
> I have a spring application, which should connect to the kafka. This is
> what my Docker file looks for spring application:
>
> ```
> FROM maven:3-jdk-11 as builder
> # create app folder for sources
> RUN mkdir -p /build
> WORKDIR /build
> COPY pom.xml /build
> #Download all required dependencies into one layer
> RUN mvn -B dependency:resolve dependency:resolve-plugins
> #Copy source code
> COPY src /build/src
> # Build application
> RUN mvn package
>
> FROM openjdk:8-jdk-alpine
> RUN addgroup -S spring && adduser -S spring -G spring
> USER spring:spring
> ARG JAR_FILE=target/*.jar
> COPY ${JAR_FILE} app.jar
> ENTRYPOINT ["java","-jar","/app.jar"]
> ```
>
> The kafka/Postgres/zookeeper everything other, comes from Docker images. So
> thought will run the application in docker compose, so that it looks like
> the following:
>
>      version: '3.1'
>      services:
>          postgres:
>              image: debezium/postgres
>              environment:
>                POSTGRES_PASSWORD: qwerty
>                POSTGRES_USER: appuser
>              volumes:
>                 - ./postgres:/data/postgres
>              ports:
>                - 6532:6532
>          zookeeper:
>              image: confluentinc/cp-zookeeper
>              ports:
>                - "2181:2181"
>              environment:
>                ZOOKEEPER_CLIENT_PORT: 2181
>          kafka:
>              image: confluentinc/cp-kafka
>              depends_on:
>                - zookeeper
>                - postgres
>              ports:
>                - "9092:9092"
>              environment:
>                KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
>                KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
>                KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
>                KAFKA_LOG_CLEANER_DELETE_RETENTION_MS: 5000
>                KAFKA_BROKER_ID: 1
>                KAFKA_MIN_INSYNC_REPLICAS: 1
>                KAFKA_ADVERTISED_HOST_NAME: kafka
>          connector:
>              image: debezium/connect:latest
>              ports:
>                - "8083:8083"
>              environment:
>                GROUP_ID: 1
>                CONFIG_STORAGE_TOPIC: my_connect_configs
>                OFFSET_STORAGE_TOPIC: my_connect_offsets
>                BOOTSTRAP_SERVERS: kafka:9092
>              depends_on:
>                - zookeeper
>                - postgres
>                - kafka
>          app-server:
>          # Configuration for building the docker image for the backend
> service
>              build:
>                context: . # Use an image built from the specified dockerfile
> in the `polling-app-server` directory.
>                dockerfile: Dockerfile
>              ports:
>                - "8080:8080" # Forward the exposed port 8080 on the
> container to port 8080 on the host machine
>              restart: always
>              depends_on:
>                - kafka
>                - zookeeper
>                - postgres
>                - connector
>              environment:
>                BOOTSTRAP_SERVERS: kafka:9092
>
> I pass an environment variable `BOOTSTRAP_SERVERS` with value as
> `kafka:9092` (since `localhost:9092` doesn't work inside my docker
> environment).
>
> And in my spring code, I get the value like the following:
>
> ```
> System.getenv("bootstrap.servers")
> // or
> System.getenv("BOOTSTRAP_SERVERS")
> ```
>
> however, when I run `docker-compose up`, I get the value as `null` for the
> above Java code. Not sure, what is the best way to get the docker resolved
> network for my kafka.
>
>   So how to fix it so that java code picks up the Kafka broker inside the
> docker environment?
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message