cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Brown (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-14451) Infinity ms Commit Log Sync
Date Fri, 01 Jun 2018 21:58:00 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-14451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16498619#comment-16498619
] 

Jason Brown edited comment on CASSANDRA-14451 at 6/1/18 9:57 PM:
-----------------------------------------------------------------

Below are branches that resolve the problem
||3.0||3.11||trunk||
|[branch|https://github.com/jasobrown/cassandra/tree/14451-3.0]|[branch|https://github.com/jasobrown/cassandra/tree/14451-3.11]|[branch|https://github.com/jasobrown/cassandra/tree/14451-trunk]|
|[utests & dtests|https://circleci.com/gh/jasobrown/workflows/cassandra/tree/14451-3.0]|[utests
& dtests|https://circleci.com/gh/jasobrown/workflows/cassandra/tree/14451-3.11]|[utests
& dtests|https://circleci.com/gh/jasobrown/workflows/cassandra/tree/14451-trunk]|

The bug was using {{markerIntervalNanos}} in the {{wakeUpAt}} variable as basis for determining
if we've lagged in the actual flush to disk. The code should use {{syncIntervalNanos}} for
that determination. Once again, the only problem here is around determining if we should log
about the commitlog flushing falling behind, not that the commitlog is actually falling behind
(it ins't, at least as far as the overlogging here is concerned).

Most of the change was moving the logging code out of the primary {{AbstractCommitLogService.SyncRunnable#sync()}}
and into a subroutine. This allowed me to add unit tests, as well as clean up/clarify the
{{sync()}} method.


was (Author: jasobrown):
Below are branches that resolve the problem

||3.0||3.11||trunk||
|[branch|https://github.com/jasobrown/cassandra/tree/14451-3.0]|[branch|https://github.com/jasobrown/cassandra/tree/14451-3.11]|[branch|https://github.com/jasobrown/cassandra/tree/14451-trunk]|
|[utests &amp; dtests|https://circleci.com/gh/jasobrown/workflows/cassandra/tree/14451-3.0]|[utests
&amp; dtests|https://circleci.com/gh/jasobrown/workflows/cassandra/tree/14451-3.11]|[utests
&amp; dtests|https://circleci.com/gh/jasobrown/workflows/cassandra/tree/14451-trunk]|
||

The bug was using {{markerIntervalNanos}} in the {{wakeUpAt}} variable as basis for determining
if we've lagged in the actual flush to disk. The code should use {{syncIntervalNanos}} for
that determination.

Most of the change was moving the logging code out of the primary {{AbstractCommitLogService.SyncRunnable#sync()}}
and into a subroutine. This allowed me to add unit tests, as well as clean up/clarify the
{{sync()}} method.

> Infinity ms Commit Log Sync
> ---------------------------
>
>                 Key: CASSANDRA-14451
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-14451
>             Project: Cassandra
>          Issue Type: Bug
>         Environment: 3.11.2 - 2 DC
>            Reporter: Harry Hough
>            Assignee: Jason Brown
>            Priority: Minor
>             Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> Its giving commit log sync warnings where there were apparently zero syncs and therefore
gives "Infinityms" as the average duration
> {code:java}
> WARN [PERIODIC-COMMIT-LOG-SYNCER] 2018-05-16 21:11:14,294 NoSpamLogger.java:94 - Out
of 0 commit log syncs over the past 0.00s with average duration of Infinityms, 1 have exceeded
the configured commit interval by an average of 74.40ms 
> WARN [PERIODIC-COMMIT-LOG-SYNCER] 2018-05-16 21:16:57,844 NoSpamLogger.java:94 - Out
of 0 commit log syncs over the past 0.00s with average duration of Infinityms, 1 have exceeded
the configured commit interval by an average of 198.69ms 
> WARN [PERIODIC-COMMIT-LOG-SYNCER] 2018-05-16 21:24:46,325 NoSpamLogger.java:94 - Out
of 0 commit log syncs over the past 0.00s with average duration of Infinityms, 1 have exceeded
the configured commit interval by an average of 264.11ms 
> WARN [PERIODIC-COMMIT-LOG-SYNCER] 2018-05-16 21:29:46,393 NoSpamLogger.java:94 - Out
of 32 commit log syncs over the past 268.84s with, average duration of 17.56ms, 1 have exceeded
the configured commit interval by an average of 173.66ms{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message