kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: Is 0.7.1 slower than 0.7.0
Date Tue, 10 Jul 2012 15:33:54 GMT
Also, did you enable compression in the producer?

Thanks,

Jun

On Mon, Jul 9, 2012 at 11:53 PM, Min <miniway@gmail.com> wrote:

> Hello,
>
> I tested again and I could reproduce the situation.
>
> I run 0.7.0 with the following shell command
> ./kafka-0.7.0-incubating-src/bin/kafka-server-start.sh
> config/server.properties
> then 0.7.1
> ./kafka-0.7.1-incubating/bin/kafka-server-start.sh config/server.properties
>
> configuration values in the server.properties are all default but the
> ZK address.
>
> My producer and broker were running on different machines in same rack.
> They are not VMs but physical machines. Also I don't feel some other
> process interfere my test.
>
>
> I'm attaching the dstat result and my producer code.
> Producer is so simple that read files in a given directory by line and
> send every 1000 lines to the broker.
>
> Thanks
> Min
>
> ./kafka-0.7.0-incubating-src/bin/kafka-server-start.sh
> config/server.properties
> (Producer)
> ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
> usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
>  23   2  72   2   0   1|  88M  896k|  98k   50M|   0     0 |2855  2627
>  20   1  77   1   0   1|  81M 1144k|  79k   48M|   0     0 |2858  2423
>  21   1  77   0   0   1|  85M    0 |  77k   47M|   0     0 |2655  2222
>   6   0  93   0   0   0|  24M  240k|  38k   15M|   0     0 |1664  1232
>  18   1  80   0   1   1|  71M    0 |  72k   42M|   0     0 |2657  2152
>  21   1  69   7   0   1|  75M 1704k|  92k   48M|   0     0 |2959  2872
>  20   2  52  25   0   1|  90M  512k|  82k   48M|   0     0 |2884  2940
>  20   1  78   0   0   0|  85M    0 |  77k   47M|   0     0 |2800  2459
>  22   2  75   0   1   1|  88M  320k|  81k   51M|   0     0 |2840  2590
>  24   1  74   0   1   0|  90M    0 |  88k   52M|   0     0 |2819  2526
>  23   1  71   2   0   1|  87M 1408k|  96k   51M|   0     0 |2924  2661
>  23   2  73   1   1   1|  89M  744k|  82k   51M|   0     0 |2851  2485
>  25   1  72   0   1   1|  94M    0 |  89k   55M|   0     0 |2941  2545
>  22   1  76   0   0   1|  88M  240k|  81k   50M|   0     0 |2804  2581
>  23   1  75   0   0   1|  93M    0 |  83k   53M|   0     0 |2868  2530
>  23   1  69   5   0   1|  87M 1400k| 100k   53M|   0     0 |2959  2668
>
> (Broker 0.7.0)
> usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
>   4   3  86   4   0   2|2056k  197M|  51M   68k|   0     0 |  11k 1911
>   5   3  86   4   0   2|2048k  209M|  53M   72k|   0     0 |  11k 2097
>   5   3  85   4   0   2|2056k  207M|  52M   77k|   0     0 |  11k 2084
>   4   4  86   4   0   2|2048k  208M|  53M   71k|   0     0 |  11k 2101
>   5   3  86   4   0   2|2064k  201M|  51M   68k|   0     0 |  11k 2055
>   4   4  86   4   0   2|2048k  207M|  53M   71k|   0     0 |  11k 2007
>   5   4  86   4   0   2|2048k  209M|  53M   71k|   0     0 |  11k 2018
>   5   3  85   4   0   2|2056k  207M|  52M   76k|   0     0 |  11k 1997
>   4   4  85   5   0   2|2056k  197M|  50M   67k|   0     0 |  11k 1907
>   5   4  86   4   0   2|2056k  205M|  52M   71k|   0     0 |  11k 2053
>   5   3  86   4   0   2|2048k  205M|  52M   70k|   0     0 |  11k 2000
>
> ./kafka-0.7.1-incubating/bin/kafka-server-start.sh config/server.properties
>
> (Producer)
> ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
> usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
>   9   1  89   1   0   0|  41M  416k|  51k   27M|   0     0 |2205  1618
>   9   1  88   2   0   0|  49M    0 |  47k   27M|   0     0 |2212  1952
>   8   1  89   1   0   0|  44M    0 |  58k   26M|   0     0 |2201  1713
>  10   1  87   2   0   0|  47M  256k|  57k   26M|   0     0 |2199  1992
>   9   1  84   5   0   1|  46M 2488k|  56k   27M|   0     0 |2472  1924
>   9   1  89   1   0   0|  42M  384k|  53k   26M|   0     0 |2187  1730
>   9   1  89   1   0   0|  48M    0 |  44k   27M|   0     0 |2189  1691
>  10   1  88   1   0   0|  45M    0 |  58k   27M|   0     0 |2205  1884
>   9   1  89   1   0   0|  48M  224k|  58k   27M|   0     0 |2225  1715
>   9   1  76  12   0   1|  44M 2496k|  57k   26M|   0     0 |2425  2133
>   9   1  88   2   0   0|  45M  584k|  51k   26M|   0     0 |2194  1685
>   9   1  89   1   0   0|  46M    0 |  43k   25M|   0     0 |2166  1727
>   9   1  90   1   0   0|  47M    0 |  54k   25M|   0     0 |2183  1629
>
> (Broker - 0.7.1)
> ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
> usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
>   9   2  86   2   0   1|2048k  103M|  26M   36k|   0     0 |6376  1849
>   8   2  86   2   0   1|2056k  104M|  26M   37k|   0     0 |6431  1945
>   9   2  86   2   0   1|2048k  101M|  26M   42k|   0     0 |6431  1884
>   9   2  87   2   0   1|2048k  104M|  26M   35k|   0     0 |6300  1561
>   9   2  86   2   0   1|2048k  104M|  27M   37k|   0     0 |6542  1543
>   9   2  86   2   0   1|2056k  104M|  26M   37k|   0     0 |6487  1638
>   9   2  87   2   0   1|2048k  101M|  26M   36k|   0     0 |6402  1721
>   9   2  86   2   0   1|2048k  104M|  26M   43k|   0     0 |6436  1762
>   9   2  86   2   0   1|2048k  106M|  27M   37k|   0     0 |6527  1799
>   9   2  86   3   0   1|2056k  101M|  26M   36k|   0     0 |6378  1521
>   9   2  86   2   0   1|2048k  104M|  27M   37k|   0     0 |6521  1568
>   9   2  87   2   0   1|2048k  104M|  26M   37k|   0     0 |6452  1708
>
>
>
>
> 2012/7/9 Jay Kreps <jay.kreps@gmail.com>:
> > Not that is known to me. Is this something you can reproduce? Are the
> > settings the same on the two brokers? Can you check iostat between the
> two
> > and see if the change is due to disk activity or cpu?
> >
> > -Jay
> >
> > On Sun, Jul 8, 2012 at 12:03 AM, Min <miniway@gmail.com> wrote:
> >
> >> Hello,
> >>
> >> Recently I've upgrade to 0.7.1.  But 0.7.1 didn't show performance as
> >> 0.7.0.
> >>
> >> With 0.7.0 I can push 45MBps to the broker. But I only can push 27MBps
> >> with 0.7.1.
> >>
> >> Is there any a fetch to affect the performance?
> >>
> >> Thanks
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message