samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mohanraj v <mohanrajv....@gmail.com>
Subject Re: How to partition a topic into multiple and how to create multiple Samza Containers
Date Fri, 18 Mar 2016 13:11:28 GMT
Hi,

     Im trying to create more than one container in my application(Single
machine).
I have 1,00,000 records in one kafka topic.How to partition it into two and
process it in parallel. I configured my job properties as below but i didnt
get multiple containers.Kindly reply me as soon as possible to work on this
application.

machine configuration:
4GB RAM,2 cores

# Job

job.factory.class=org.apache.samza.job.yarn.YarnJobFactory
job.name=job-parser

# YARN
yarn.package.path=file:///home/hello-samza/target/hello-samza-0.10.0-dist.tar.gz
yarn.container.count=2
yarn.container.memory.mb=512
yarn.container.cpu.cores=2
#yarn.am.container.memory.mb=1024

# Task
task.class=samza.task.ParserStreamTask
task.inputs=kafka.input

# Serializers
serializers.registry.string.class=org.apache.samza.serializers.StringSerdeFactory

# Kafka System
systems.kafka.samza.factory=org.apache.samza.system.kafka.KafkaSystemFactory
systems.kafka.samza.msg.serde=string
systems.kafka.consumer.zookeeper.connect=localhost:2181/
systems.kafka.producer.bootstrap.servers=localhost:9092

# Job Coordinator
job.coordinator.system=kafka
job.coordinator.replication.factor=1



Thanks,
Mohan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message