lucene-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 张海雷 <zhanghai...@bytedance.com>
Subject Fwd: High concurrency scenarios IndexReader‘s registerParentReader method takes a lot of CPU time
Date Thu, 16 Apr 2020 14:27:42 GMT


> 
> Hi, all
> In our Elastic Search benchmark, found out that  IndexReader‘s registerParentReader
method takes a lot of CPU time  
> Environment:
> ES Version: 7.5.1
> Lucene Version: 8.3.0
> 
> Index:
> Shard Num 1
> Replicator’s Num:1
> Segment Count: 538
> Size:49.5GB
> 
> Benchmark result: qps:5200,pct99 600ms
> CPU Util:93%
> 
> Use flame graph find the hot method  IndexReader‘s registerParentReader. dive into
code find out that Have a  race condition that on the synchronizedSet Because of only one
> Shard and one shard have too many segments, this intensify competition
> 
> 
> 
> 
> To verify this assumption,execute force merge on this index,after force merge:
> QPS: more then 5600
> PCT99: 60ms
> CPU Util: 11% 
> The performance improved by a factor of 10x
> 
> So we can use ConcurrentWeakKeyHashmap instead of synchronizedSet  to reduce lock competition?
> 
> 
> 
> 
> 


Mime
View raw message