giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hassan Eslami (JIRA)" <>
Subject [jira] [Commented] (GIRAPH-462) Multithreading breaks out-of-core graph
Date Fri, 20 May 2016 22:39:12 GMT


Hassan Eslami commented on GIRAPH-462:

The new out-of-core feature has been pushed to the trunk (GIRAPH-1066). This issue should
be resolved, although out-of-core optimization and its integration with checkpointing is still

> Multithreading breaks out-of-core graph
> ---------------------------------------
>                 Key: GIRAPH-462
>                 URL:
>             Project: Giraph
>          Issue Type: Bug
>            Reporter: Alessandro Presta
>            Priority: Critical
>         Attachments: GIRAPH-461.patch
> [~cmartella] pointed out this issue: when using multithreaded computation in conjunction
with out-of-core graph, we incur in a race condition. The compute threads share the same DiskBackedPartitionStore,
whose getPartition() method is not meant to be thread-safe. When two threads request two out-of-core
partitions concurrently, they both try to load it to the same slot.
> The result is that we can lose the reference to one of the two partitions (which will
not be written back to disk) and we can incur in a NullPointerException when both threads
are trying to offload the currently loaded partition to disk.
> I ran this test to confirm the issue:
> All tests pass except the one that uses both out-of-core graph and multiple compute threads.
> The error is the following:

This message was sent by Atlassian JIRA

View raw message