jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrei Dulceanu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (OAK-6678) Syncing big blobs fails since StandbyServer sends persisted head
Date Wed, 27 Sep 2017 13:52:00 GMT

    [ https://issues.apache.org/jira/browse/OAK-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16182596#comment-16182596
] 

Andrei Dulceanu commented on OAK-6678:
--------------------------------------

[~frm], one more thing that I forgot talking in my previous comment:

{quote}
The timeout when reading the head state must not come from the standby. We risk unwanted denial
of service on the primary if a standby specifies too high timeouts in the requests.
{quote}
I think the DoS on the primary can't actually happen due to a high timeout used in the client.
That timeout is not going to be used to the fullest, but once a persisted head will be available
on the primary, the head record id will be sent right away to the standby. IIUC, this situation
can happen only once, when the primary is started and should last for about 5s, until the
flush thread kicks in. 

Now let's suppose that we re-use the default timeout from the client, namely 60s. If the primary
is unable to persist its first state in 60s, then I think something really fishy is happening
on the instance, don't you agree?

> Syncing big blobs fails since StandbyServer sends persisted head
> ----------------------------------------------------------------
>
>                 Key: OAK-6678
>                 URL: https://issues.apache.org/jira/browse/OAK-6678
>             Project: Jackrabbit Oak
>          Issue Type: Bug
>          Components: segment-tar, tarmk-standby
>            Reporter: Andrei Dulceanu
>            Assignee: Andrei Dulceanu
>              Labels: cold-standby, resilience
>             Fix For: 1.8, 1.7.9
>
>         Attachments: OAK-6678-02.patch, OAK-6678.patch
>
>
> With changes for OAK-6653 in place, {{ExternalPrivateStoreIT#testSyncBigBlog}} and sometimes
{{ExternalSharedStoreIT#testSyncBigBlob}} are failing on CI:
> {noformat}
> org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT
> testSyncBigBlob(org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT)  Time
elapsed: 96.82 sec  <<< FAILURE!
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : {
} }>
> ...
> testSyncBigBlob(org.apache.jackrabbit.oak.segment.standby.ExternalPrivateStoreIT)  Time
elapsed: 95.254 sec  <<< FAILURE!
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : {
} }>
> {noformat}
> Partial stacktrace:
> {noformat}
> 14:09:08.355 DEBUG [main] StandbyServer.java:242            Binding was successful
> 14:09:08.358 DEBUG [standby-1] GetHeadRequestEncoder.java:33 Sending request from client
Bar for current head
> 14:09:08.359 DEBUG [primary-1] ClientFilterHandler.java:53  Client /127.0.0.1:52988 is
allowed
> 14:09:08.360 DEBUG [primary-1] RequestDecoder.java:42       Parsed 'get head' message
> 14:09:08.360 DEBUG [primary-1] CommunicationObserver.java:79 Message 'get head' received
from client Bar
> 14:09:08.362 DEBUG [primary-1] GetHeadRequestHandler.java:43 Reading head for client
Bar
> 14:09:08.363 WARN  [primary-1] ExceptionHandler.java:31     Exception caught on the server
> java.lang.NullPointerException: null
> 	at org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyHeadReader.readHeadRecordId(DefaultStandbyHeadReader.java:32)
~[oak-segment-tar-1.8-SNAPSHOT.jar:1.8-SNAPSHOT]
> 	at org.apache.jackrabbit.oak.segment.standby.server.GetHeadRequestHandler.channelRead0(GetHeadRequestHandler.java:45)
~[oak-segment-tar-1.8-SNAPSHOT.jar:1.8-SNAPSHOT]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message