kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 刘明敏 <diveintotomor...@gmail.com>
Subject Re: kafka-dev - Google Groups: Message Pending [{IK_6mr_o_d-UVSoCaWcwABV4kOW3nBaU0}]
Date Wed, 28 Mar 2012 10:57:13 GMT
4 cpus on Procuder:

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 30
model name : Intel(R) Xeon(R) CPU           X3430  @ 2.40GHz
stepping : 5
cpu MHz : 2393.580
cache size : 8192 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36
clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm
constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc
aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips : 4787.16
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 30
model name : Intel(R) Xeon(R) CPU           X3430  @ 2.40GHz
stepping : 5
cpu MHz : 2393.580
cache size : 8192 KB
physical id : 0
siblings : 4
core id : 1
cpu cores : 4
apicid : 2
initial apicid : 2
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36
clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm
constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc
aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips : 4787.17
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 30
model name : Intel(R) Xeon(R) CPU           X3430  @ 2.40GHz
stepping : 5
cpu MHz : 2393.580
cache size : 8192 KB
physical id : 0
siblings : 4
core id : 2
cpu cores : 4
apicid : 4
initial apicid : 4
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36
clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm
constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc
aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips : 4787.17
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 30
model name : Intel(R) Xeon(R) CPU           X3430  @ 2.40GHz
stepping : 5
cpu MHz : 2393.580
cache size : 8192 KB
physical id : 0
siblings : 4
core id : 3
cpu cores : 4
apicid : 6
initial apicid : 6
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36
clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm
constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc
aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips : 4787.17
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

===
not sure if I am sending the right place this time.

2012/3/28 Jay Kreps <jay.kreps@gmail.com>

> What does CPU look like on the producer?
>
> Also, can you please use the apache mailing list (cc'd). This google group
> is  from our pre-apache days and is long dead.
>
> -Jay
>
> On Tue, Mar 27, 2012 at 7:42 PM, Kafka
> <kafka-dev+msgappr@googlegroups.com>wrote:
>
> >   A message has been sent to the kafka-dev<
> http://groups.google.com/group/kafka-dev?hl=en_US>group and is awaiting
> approval. We've included this message for your review.
> >
> > The message requires moderation because the user does not have permission
> > to post.
> >
> > You can approve or reject<
> http://groups.google.com/group/kafka-dev/pendmsg?hl=en_US>this message or
> you can approve this message by replying to this email.
> >  [image: Visit Google Groups]
> >
> > Start <https://groups.google.com/groups/create?hl=en_US&src=email> your
> > own group, visit <http://groups.google.com/support/?hl=en_US&src=email>the
> help center, or
> > report <https://groups.google.com/groups/abuse?hl=en_US&src=email>
> abuse.
> >
> >
> > ---------- Forwarded message ----------
> > From: 刘明敏 <diveintotomorrow@gmail.com>
> > To: kafka-users@incubator.apache.org
> > Cc: kafka-dev@googlegroups.com
> > Date: Wed, 28 Mar 2012 10:42:11 +0800
> > Subject: Re: [kafka-dev] performance of async producer stops increase
> when
> > broker is more than 2
> > I have 1 producer on 1 machine.
> >
> > The perf of sync producer is :
> >
> > 1 borker:
> >    *20.3277(M/s)*,*213151.4441(#/s)* (sync)
> > 2 borkers:
> >    *39.5995*, *415230.6606* (sync)
> > 3 borkers:
> >    *58.0836*, *609050.4903*(sync)
> >
> > compared with async producer :
> >  when 1 broker:
> >  20.3164 M/s,213033.3823#/s (async)
> >
> >  when 2 brokers:
> >  37.4754 M/s,392958.1892#/s (async)
> >
> >  when 3 brokers(perf stops increase)
> >  36.6925 M/s, 384748.5668#/s (async)
> >
> > the sync producer reached 600K #/s(far beyond sync producer's 384k #/s)
> So
> > I doubt if it is the case of maxing out network or procuder.
> >
> > ===
> > and,I post the thread from google group:
> > https://groups.google.com/forum/?fromgroups#!forum/kafka-dev
> >
> > On Wed, Mar 28, 2012 at 3:25 AM, Jay Kreps <jay.kreps@gmail.com> wrote:
> >
> >> Hi,
> >>
> >> How many producers are you using and on how many machines? Also what is
> >> the
> >> network speed (1gbit?) Likely you are maxing out either the producer
> >> machine(s) or the network itself.
> >>
> >> Also, quick question, how did you come across this address? This address
> >> has been dead for a while now, and the new mailing list is on Apache so
> I
> >> just want to make sure we don't have a dangling pointer somewhere. Would
> >> be
> >> good to move the discussion to the Apache list so others can find it.
> >>
> >> -Jay
> >>
> >> On Tue, Mar 27, 2012 at 2:10 AM, mmLiu <diveintotomorrow@gmail.com>
> >> wrote:
> >>
> >> > I tested the performance of producer with
> >> ./bin/kafka-producer-perf-test.sh
> >> >
> >> > when 1 broker:
> >> > 20.3164 M/s,213033.3823#/s (async)
> >> >
> >> > when 2 brokers:
> >> > 37.4754 M/s,392958.1892#/s (async)
> >> >
> >> > when 3 brokers(perf stops increase)
> >> > 36.6925 M/s, 384748.5668#/s (async)
> >> >
> >> > when 4 brokers(even decrease):
> >> > 33.2244 M/s, 348383.5006 #/s (async)
> >> >
> >> >
> >> > Any idea why?
> >> >
> >> > FYI:the performance of sync producer increases(almost) linearly as
> >> number
> >> > of brokers increases.
> >>
> >
> >
> >
> > --
> > Best Regards
> >
> > ----------------------
> > 刘明敏 | mmLiu
> >
> >
> >
>



-- 
Best Regards

----------------------
刘明敏 | mmLiu

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message