cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Stupp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
Date Tue, 09 Dec 2014 22:41:14 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240193#comment-14240193
] 

Robert Stupp commented on CASSANDRA-7438:
-----------------------------------------

bq. Lot's of cool stuff

Thx :)

Unit testing: you are absolutely right. (Will go on with that next)

bq. unit test that fuzzes against a parallel implementation at the same time using a different
LRU map implementation 

Do you mean sth. like LinkedHashMap with removeEldestEntry() ? It's some effort to get a "nice"
implementation for unit tests - but, yeah - makes sense.

bq. duplicates reference counting code

removed duplicated code

bq. cleanup/eviction code go away ... remove entries until it [fits]

much easier ; cleaner code ; implemented - but not completely sold on the new implementation
yet (quick hack yet)

bq. C* cache interface ... get compression and more efficient IO [later]

That's fair. I just saw some minutes ago that row-cache serialization "only" persists the
keys and not the values - so the existing implementation in OHC would need to be changed /
extended / whatever. I thought it persists the value, too.

> Serializing Row cache alternative (Fully off heap)
> --------------------------------------------------
>
>                 Key: CASSANDRA-7438
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7438
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>         Environment: Linux
>            Reporter: Vijay
>            Assignee: Vijay
>              Labels: performance
>             Fix For: 3.0
>
>         Attachments: 0001-CASSANDRA-7438.patch, tests.zip
>
>
> Currently SerializingCache is partially off heap, keys are still stored in JVM heap as
BB, 
> * There is a higher GC costs for a reasonably big cache.
> * Some users have used the row cache efficiently in production for better results, but
this requires careful tunning.
> * Overhead in Memory for the cache entries are relatively high.
> So the proposal for this ticket is to move the LRU cache logic completely off heap and
use JNI to interact with cache. We might want to ensure that the new implementation match
the existing API's (ICache), and the implementation needs to have safe memory access, low
overhead in memory and less memcpy's (As much as possible).
> We might also want to make this cache configurable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message