kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 刘明敏 <diveintotomor...@gmail.com>
Subject Re: kafka-dev - Google Groups: Message Pending [{IK_6mr_o_d-UVSoCaWcwABV4kOW3nBaU0}]
Date Thu, 29 Mar 2012 03:34:51 GMT
wow,seems that's the case:

when running async producer,the cpu utilization is like this( I use vmstat
to check the cpu usage) :

procs -----------memory---------- ---swap-- -----io---- --system--
-----cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy idwa st
29  0  37472 336768  73560 2555980    0    0     0  2224 9137 424623 79 20
 1  0  0
21  0  37472 1260664  62384 1608728    0    0     0  4412 17336 422321 53
29 16  1  0
14  0  37472 1252836  62424 1612268    0    0     0  7720 11486 1043389 57
40  3  0  0
12  0  37472 1255632  62496 1621888    0    0     0  9920 10556 1394823 53
46  1  0  0
13  2  37472 1259244  62548 1631272    0    0     0  9920 10335 1383344 51
47  2  0  0
11  0  37472 1267484  61660 1601680    0    0     0  9932 16050 1360485 49
48  4  0  0
14  1  37472 1257632  61732 1611536    0    0     0 10460 9847 1431470 51
48  1  0  0
12  0  37472 1247524  61800 1620844    0    0     0  8804 13658 1191618 54
42  4  0  0
13  0  37472 1239464  61868 1631208    0    0     0 10984 14603 1443709 50
48  2  1  0
15  2  37472 1230156  61936 1641628    0    0     0 10980 11569 1480576 49
50  1  0  0
12  0  37472 1220236  61992 1652180    0    0     0  9912 11564 1475147 50
49  1  1  0
12  1  37472 1210812  62056 1662000    0    0     0 10964 10469 1455622 49
49  2  0  0
13  0  37472 1201636  62132 1672208    0    0     0  9904 12127 1467242 49
49  1  0  0
12  2  37472 1191964  62208 1682308    0    0     0 10980 12817 1448184 50
50  1  0  0
12  0  37472 1181796  62264 1692868    0    0     0  9884 15399 1452468 49
49  2  0  0
14  0  37472 1179980  61824 1695388    0    0     0 10988 14860 1303783 50
46  3  1  0
12  2  37472 1202232  59256 1653940    0    0     0 10984 14979 1344551 47
52  2  0  0
12  0  37472 1191568  59320 1664252    0    0     0  9884 10081 1488794 49
51  1  0  0
14  2  37472 1181276  59396 1674340    0    0     0 10984 12535 1433143 48
48  3  1  0
16  1  37472 1170788  59468 1685128    0    0     0 10960 11839 1468341 48
50  2  0  0
13  0  37472 1161612  59548 1695324    0    0     0  9968 15067 1467583 48
50  1  0  0
11  2  37472 1151940  59624 1705468    0    0     0 10984 13112 1482698 48
51  1  0  0
15  2  37472 1141524  59696 1716104    0    0     0 10984 14507 1444776 50
48  2  1  0
 4  2  37472 1152144  57940 1677388    0    0     0 10992 13915 1029274 51
39  9  2  0

while the sync producer's cpu usage is much lower,like this:

procs -----------memory---------- ---swap-- -----io---- --system--
-----cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
st
 1  2  37472 1148876  52496 1729876    0    0     0 21964 9423 3919 29
 5 4521  0
 0  2  37472 1130836  52652 1751268    0    0     0 21976 7931 4185 15
 3 5625  0
 0  2  37472 1112360  52820 1774020    0    0     0 23104 7275 4437 14
 3 5032  0
 0  2  37472 1094876  52984 1796472    0    0     0 23060 7201 3801 13  4
53 31  0
 3  2  37472 1078572  53140 1819200    0    0     0 23064 6965 4418 13  4
57 26  0
 4  0  37472 1061400  53288 1840004    0    0     0 20864 7027 3930 12  3
58 26  0
 3  0  37472 1043916  53444 1861068    0    0     0 21964 6957 4379 14  3
61 21  0
 3  2  37472 1025324  53616 1883332    0    0     0 23068 7062 4104 14  4
64 18  0
 4  2  37472 1008708  53764 1904044    0    0     0 20868 6246 3802 12  3
57 29  0

can you help me understand why the async producer is much more cpu
extensive than the sync procuder ?

On Thu, Mar 29, 2012 at 12:04 AM, Jay Kreps <jay.kreps@gmail.com> wrote:

> Ah, but what is the cpu utilization? Can you run "top" or something during
> the test? My suspicion is that the producer is maybe just cpu bound.
>
> -Jay
>
> On Wed, Mar 28, 2012 at 3:57 AM, 刘明敏 <diveintotomorrow@gmail.com> wrote:
>
> > 4 cpus on Procuder:
> >
> > processor : 0
> > vendor_id : GenuineIntel
> > cpu family : 6
> > model : 30
> > model name : Intel(R) Xeon(R) CPU           X3430  @ 2.40GHz
> > stepping : 5
> > cpu MHz : 2393.580
> > cache size : 8192 KB
> > physical id : 0
> > siblings : 4
> > core id : 0
> > cpu cores : 4
> > apicid : 0
> > initial apicid : 0
> > fpu : yes
> > fpu_exception : yes
> > cpuid level : 11
> > wp : yes
> > flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
> pse36
> > clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm
> > constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc
> > aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm
> > sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
> > bogomips : 4787.16
> > clflush size : 64
> > cache_alignment : 64
> > address sizes : 36 bits physical, 48 bits virtual
> > power management:
> >
> > processor : 1
> > vendor_id : GenuineIntel
> > cpu family : 6
> > model : 30
> > model name : Intel(R) Xeon(R) CPU           X3430  @ 2.40GHz
> > stepping : 5
> > cpu MHz : 2393.580
> > cache size : 8192 KB
> > physical id : 0
> > siblings : 4
> > core id : 1
> > cpu cores : 4
> > apicid : 2
> > initial apicid : 2
> > fpu : yes
> > fpu_exception : yes
> > cpuid level : 11
> > wp : yes
> > flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
> pse36
> > clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm
> > constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc
> > aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm
> > sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
> > bogomips : 4787.17
> > clflush size : 64
> > cache_alignment : 64
> > address sizes : 36 bits physical, 48 bits virtual
> > power management:
> >
> > processor : 2
> > vendor_id : GenuineIntel
> > cpu family : 6
> > model : 30
> > model name : Intel(R) Xeon(R) CPU           X3430  @ 2.40GHz
> > stepping : 5
> > cpu MHz : 2393.580
> > cache size : 8192 KB
> > physical id : 0
> > siblings : 4
> > core id : 2
> > cpu cores : 4
> > apicid : 4
> > initial apicid : 4
> > fpu : yes
> > fpu_exception : yes
> > cpuid level : 11
> > wp : yes
> > flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
> pse36
> > clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm
> > constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc
> > aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm
> > sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
> > bogomips : 4787.17
> > clflush size : 64
> > cache_alignment : 64
> > address sizes : 36 bits physical, 48 bits virtual
> > power management:
> >
> > processor : 3
> > vendor_id : GenuineIntel
> > cpu family : 6
> > model : 30
> > model name : Intel(R) Xeon(R) CPU           X3430  @ 2.40GHz
> > stepping : 5
> > cpu MHz : 2393.580
> > cache size : 8192 KB
> > physical id : 0
> > siblings : 4
> > core id : 3
> > cpu cores : 4
> > apicid : 6
> > initial apicid : 6
> > fpu : yes
> > fpu_exception : yes
> > cpuid level : 11
> > wp : yes
> > flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
> pse36
> > clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm
> > constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc
> > aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm
> > sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
> > bogomips : 4787.17
> > clflush size : 64
> > cache_alignment : 64
> > address sizes : 36 bits physical, 48 bits virtual
> > power management:
> >
> > ===
> > not sure if I am sending the right place this time.
> >
> > 2012/3/28 Jay Kreps <jay.kreps@gmail.com>
> >
> > > What does CPU look like on the producer?
> > >
> > > Also, can you please use the apache mailing list (cc'd). This google
> > group
> > > is  from our pre-apache days and is long dead.
> > >
> > > -Jay
> > >
> > > On Tue, Mar 27, 2012 at 7:42 PM, Kafka
> > > <kafka-dev+msgappr@googlegroups.com>wrote:
> > >
> > > >   A message has been sent to the kafka-dev<
> > > http://groups.google.com/group/kafka-dev?hl=en_US>group and is
> awaiting
> > > approval. We've included this message for your review.
> > > >
> > > > The message requires moderation because the user does not have
> > permission
> > > > to post.
> > > >
> > > > You can approve or reject<
> > > http://groups.google.com/group/kafka-dev/pendmsg?hl=en_US>this message
> > or
> > > you can approve this message by replying to this email.
> > > >  [image: Visit Google Groups]
> > > >
> > > > Start <https://groups.google.com/groups/create?hl=en_US&src=email>
> > your
> > > > own group, visit <
> http://groups.google.com/support/?hl=en_US&src=email
> > >the
> > > help center, or
> > > > report <https://groups.google.com/groups/abuse?hl=en_US&src=email>
> > > abuse.
> > > >
> > > >
> > > > ---------- Forwarded message ----------
> > > > From: 刘明敏 <diveintotomorrow@gmail.com>
> > > > To: kafka-users@incubator.apache.org
> > > > Cc: kafka-dev@googlegroups.com
> > > > Date: Wed, 28 Mar 2012 10:42:11 +0800
> > > > Subject: Re: [kafka-dev] performance of async producer stops increase
> > > when
> > > > broker is more than 2
> > > > I have 1 producer on 1 machine.
> > > >
> > > > The perf of sync producer is :
> > > >
> > > > 1 borker:
> > > >    *20.3277(M/s)*,*213151.4441(#/s)* (sync)
> > > > 2 borkers:
> > > >    *39.5995*, *415230.6606* (sync)
> > > > 3 borkers:
> > > >    *58.0836*, *609050.4903*(sync)
> > > >
> > > > compared with async producer :
> > > >  when 1 broker:
> > > >  20.3164 M/s,213033.3823#/s (async)
> > > >
> > > >  when 2 brokers:
> > > >  37.4754 M/s,392958.1892#/s (async)
> > > >
> > > >  when 3 brokers(perf stops increase)
> > > >  36.6925 M/s, 384748.5668#/s (async)
> > > >
> > > > the sync producer reached 600K #/s(far beyond sync producer's 384k
> #/s)
> > > So
> > > > I doubt if it is the case of maxing out network or procuder.
> > > >
> > > > ===
> > > > and,I post the thread from google group:
> > > > https://groups.google.com/forum/?fromgroups#!forum/kafka-dev
> > > >
> > > > On Wed, Mar 28, 2012 at 3:25 AM, Jay Kreps <jay.kreps@gmail.com>
> > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> How many producers are you using and on how many machines? Also what
> > is
> > > >> the
> > > >> network speed (1gbit?) Likely you are maxing out either the producer
> > > >> machine(s) or the network itself.
> > > >>
> > > >> Also, quick question, how did you come across this address? This
> > address
> > > >> has been dead for a while now, and the new mailing list is on Apache
> > so
> > > I
> > > >> just want to make sure we don't have a dangling pointer somewhere.
> > Would
> > > >> be
> > > >> good to move the discussion to the Apache list so others can find
> it.
> > > >>
> > > >> -Jay
> > > >>
> > > >> On Tue, Mar 27, 2012 at 2:10 AM, mmLiu <diveintotomorrow@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > I tested the performance of producer with
> > > >> ./bin/kafka-producer-perf-test.sh
> > > >> >
> > > >> > when 1 broker:
> > > >> > 20.3164 M/s,213033.3823#/s (async)
> > > >> >
> > > >> > when 2 brokers:
> > > >> > 37.4754 M/s,392958.1892#/s (async)
> > > >> >
> > > >> > when 3 brokers(perf stops increase)
> > > >> > 36.6925 M/s, 384748.5668#/s (async)
> > > >> >
> > > >> > when 4 brokers(even decrease):
> > > >> > 33.2244 M/s, 348383.5006 #/s (async)
> > > >> >
> > > >> >
> > > >> > Any idea why?
> > > >> >
> > > >> > FYI:the performance of sync producer increases(almost) linearly
as
> > > >> number
> > > >> > of brokers increases.
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Best Regards
> > > >
> > > > ----------------------
> > > > 刘明敏 | mmLiu
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Best Regards
> >
> > ----------------------
> > 刘明敏 | mmLiu
> >
>



-- 
Best Regards

----------------------
刘明敏 | mmLiu

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message