lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amrit Sarkar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-11200) provide a config to enable disable ConcurrentMergeSchedule.doAutoIOThrottle
Date Sat, 05 Aug 2017 09:23:00 GMT

    [ https://issues.apache.org/jira/browse/SOLR-11200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115348#comment-16115348
] 

Amrit Sarkar commented on SOLR-11200:
-------------------------------------

Ah! I don't this will serve our purpose in bulk indexing, logs :: 

{code}
mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=5, maxMergeCount=15, ioThrottle=false
2017-08-05 09:14:03.005 INFO  (qtp1205044462-19) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1]
o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-19]: updateMergeThreads ioThrottle=false
targetMBPerSec=10240.0 MB/sec
mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=5, maxMergeCount=15, ioThrottle=false
2017-08-05 09:15:51.196 INFO  (qtp1205044462-69) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1]
o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-69]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:15:56.711 INFO  (Lucene Merge Thread #0) [c:collection1 s:shard1 r:core_node2
x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][Lucene Merge Thread
#0]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:10.752 INFO  (qtp1205044462-17) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1]
o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-17]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:18.229 INFO  (Lucene Merge Thread #1) [c:collection1 s:shard1 r:core_node2
x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][Lucene Merge Thread
#1]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:26.516 INFO  (qtp1205044462-69) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1]
o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-69]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:35.551 INFO  (Lucene Merge Thread #2) [c:collection1 s:shard1 r:core_node2
x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][Lucene Merge Thread
#2]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:38.580 INFO  (qtp1205044462-18) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1]
o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-18]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:49.397 INFO  (Lucene Merge Thread #3) [c:collection1 s:shard1 r:core_node2
x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][Lucene Merge Thread
#3]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:56.630 INFO  (qtp1205044462-15) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1]
o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-15]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
{code}

See the {{targetMBPerSec}} is initialised to {{10gbps}}, but then it falls back to default
{{20mbps}}, instead of maintaining at 10gbps. Maybe {{SolrIndexConfig#buildMergeSchedule}}
is not the right place to do it. I will look more.

> provide a config to enable disable ConcurrentMergeSchedule.doAutoIOThrottle
> ---------------------------------------------------------------------------
>
>                 Key: SOLR-11200
>                 URL: https://issues.apache.org/jira/browse/SOLR-11200
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Nawab Zada Asad iqbal
>            Priority: Minor
>         Attachments: SOLR-11200.patch, SOLR-11200.patch
>
>
> This config can be useful while bulk indexing. Lucene introduced it https://issues.apache.org/jira/browse/LUCENE-6119
. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message