kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ewen Cheslack-Postava <e...@confluent.io>
Subject Re: Kafka System test
Date Fri, 23 Jan 2015 21:22:06 GMT
1. Except for that hostname setting being a list instead of a single host,
the changes look reasonable. That is where you want to customize settings
for your setup.

2 & 3. Yes, you'll want to update those files as well. They top-level ones
provide defaults, the ones in specific test directories provide overrides
for that specific test. But they aren't combined in any way, i.e. the more
specific one is just taken as a whole rather than being like a diff, so you
do have to update both.

You might want to take a look at
https://issues.apache.org/jira/browse/KAFKA-1748. Currently if you want to
run all tests it's a pain to change the hosts they're running on since it
requires manually editing all those files. The patch gets rid of
cluster_config.json and provides a couple of different ways of configuring
the cluster -- run everything on localhost, get cluster info from a single
json file, or get the ssh info from Vagrant.



On Fri, Jan 23, 2015 at 11:50 AM, Sa Li <salicn@gmail.com> wrote:

> Hi, All
>
> From my last ticket (Subject: kafka production server test), Guozhang
> kindly point me the system test package come with kafka source build which
> is really cool package. I took a look at this package, things are clear is
> I run it on localhost, I don't need to change anything, say,
> cluster_config.json defines entities, and system test reads
> testcase_xxxx_properties.json to override the properties in
> cluster_config.json. For example, cluster_config.json defaults hostname as
> localhost, and three brokers, I assume it will create 3 brokers in
> localhost and continue the test.
>
> Currently I install the package on a vagrant VM, and like to run the system
> test on VM and remotely access production to test production cluster. The
> production cluster has 3 nodes. kafka production cluster is on top of a
> 5-node zookeeper ensemble.  My questions is how to effectively change the
> properties on vagrant system test package.
>
> 1. change on cluster_config.json, like
>     {
>             "entity_id": "0",
>             "hostname":
> "10.100.70.28,10.100.70.29,10.100.70.30,10.100.70.31,10.100.70.32",
>             "role": "zookeeper",
>             "cluster_name": "target",
>             "kafka_home": "/etc/kafka",
>             "java_home": "/usr/lib/jvm/java-7-openjdk-amd64/jre",
>             "jmx_port": "9990"
>         },
>         {
>             "entity_id": "1",
>             "hostname": "10.100.70.28",
>             "role": "broker",
>             "cluster_name": "target",
>             "kafka_home": "/etc/kafka",
>             "java_home": "/usr/lib/jvm/java-7-openjdk-amd64/jre",
>             "jmx_port": "9991"
>         },
>
>  Here because I want to test remote servers, so I need to change the
> cluster_name as "target", right?
>
> 2.  In directory ./replication_testsuite/config/ , for all the properties
> files, do I need to change them all to be the same as the properties on
> production servers?
>
> 3. in ./replication_testsuite/testcase_xxxx/, seems I need to make
> corresponding changes as well to keep consistent with
> ./config/....properties, such as
> "log.dir": "/tmp/kafka_server_1_logs" will be change to the log.dir in my
> production server.properties, is that right?
>
>
> Hope someone who has done the system test on remote server can share some
> experience, thanks
>
>
>
> AL
>
> --
>
> Alec Li
>



-- 
Thanks,
Ewen

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message