ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxim Muzafarov <mmu...@apache.org>
Subject ExecutorServices hide assertions without logging and node stop
Date Mon, 04 May 2020 16:15:21 GMT
Igniters,


I've spent a couple of days in debug mode and found that some of the
configured ExecutorServices of an Ignite instance completely hide
assertion errors without any logging and node killing. This may
violate some internal guarantees and hide serious errors.

Let's consider, for instance, GridDhtPartitionsExchangeFuture [1]. It
has three places of submitting some Runnable on system executor
service. If an assertion error (or even any uncached exception) occurs
in the code block below it will be swallowed without logging, exchange
future completion or node stoping.

cctx.kernalContext().getSystemExecutorService().submit(new Runnable() {
    @Override public void run() {
        sendPartitions(newCrd);
    }
});

I've checked that these executor services and most of them configured
to catch only OutOfMemoryError.

Should we consider catching AssertionErrors as well and treat them as
CRITICAL_ERRORS for the Failure Handler?
Should we log uncached errors on each of them?


The most important list of executor services configured with OOM handler only:
execSvc
svcExecSvc
sysExecSvc
p2pExecSvc
restExecSvc
utilityCacheExecSvc
affExecSvc
qryExecSvc

[1] https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java#L4848

Mime
View raw message