cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vijay (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-3997) Make SerializingCache Memory Pluggable
Date Mon, 02 Apr 2012 01:00:59 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243924#comment-13243924
] 

Vijay commented on CASSANDRA-3997:
----------------------------------

Hi Pavel, 
Looks like howard malloc can work seamlessly with LD_PRELOAD but JEmalloc doesn't work well
with LD_PRELOAD.

{quote}
Jason says: The crash in free() is the only one that tells me anything at all, and my only
guesses are 1) mixed allocator usage or 2) application error, e.g. double free().  I really
don't know anything about how the JVM is structured internally , how it interacts with malloc,
how it uses/abuses dlopen(), etc., so I'm not going to be of much help without a lot more
background information.
{quote}

In other hand the attached patch avoids the crashes.

Comparision of the mallocs:

{code}
[vijay_tcasstest@vijay_tcass-i-08e1f16c java]$ java -cp /apps/nfcassandra_server/lib/concurrentlinkedhashmap-lru-1.2.jar:/apps/nfcassandra_server/lib/jna-3.3.0.jar:/apps/nfcassandra_server/lib/apache-cassandra-1.1.0-beta2-SNAPSHOT.jar:.
com.sun.jna.MallocAllocator 50000 2000000
             total       used       free     shared    buffers     cached
Mem:      71688220   10569792   61118428          0     146360    1864972
-/+ buffers/cache:    8558460   63129760
Swap:            0          0          0
**** Starting Test! ****
Total bytes read: 101423216640
Time taken: 28587
             total       used       free     shared    buffers     cached
Mem:      71688220   15950408   55737812          0     146360    1865184
-/+ buffers/cache:   13938864   57749356
Swap:            0          0          0
**** ending Test!**** 

[vijay_tcasstest@vijay_tcass-i-08e1f16c java]$ export LD_LIBRARY_PATH=/home/vijay_tcasstest/howard/
[vijay_tcasstest@vijay_tcass-i-08e1f16c java]$ java -Djava.library.path=/home/vijay_tcasstest/howard/
-cp /apps/nfcassandra_server/lib/concurrentlinkedhashmap-lru-1.2.jar:/apps/nfcassandra_server/lib/jna-3.3.0.jar:/apps/nfcassandra_server/lib/apache-cassandra-1.1.0-beta2-SNAPSHOT.jar:.
com.sun.jna.HowardMallocAllocator 50000 2000000
             total       used       free     shared    buffers     cached
Mem:      71688220   10573476   61114744          0     146320    1864972
-/+ buffers/cache:    8562184   63126036
Swap:            0          0          0
**** Starting Test! ****
Total bytes read: 101366196224
Time taken: 33959
             total       used       free     shared    buffers     cached
Mem:      71688220   16292664   55395556          0     146320    1865184
-/+ buffers/cache:   14281160   57407060
Swap:            0          0          0
**** ending Test!**** 

[vijay_tcasstest@vijay_tcass-i-08e1f16c java]$ export LD_LIBRARY_PATH=/home/vijay_tcasstest/jemalloc/lib/
[vijay_tcasstest@vijay_tcass-i-08e1f16c java]$ java -Djava.library.path=/home/vijay_tcasstest/jemalloc/lib/
-cp /apps/nfcassandra_server/lib/concurrentlinkedhashmap-lru-1.2.jar:/apps/nfcassandra_server/lib/jna-3.3.0.jar:/apps/nfcassandra_server/lib/apache-cassandra-1.1.0-beta2-SNAPSHOT.jar:.
com.sun.jna.JEMallocAllocator 50000 2000000
             total       used       free     shared    buffers     cached
Mem:      71688220   10572896   61115324          0     146332    1864972
-/+ buffers/cache:    8561592   63126628
Swap:            0          0          0
**** Starting Test! ****
Total bytes read: 101360272384
Time taken: 29310
             total       used       free     shared    buffers     cached
Mem:      71688220   13243604   58444616          0     146340    1865184
-/+ buffers/cache:   11232080   60456140
Swap:            0          0          0
**** ending Test!**** 
{code}


                
> Make SerializingCache Memory Pluggable
> --------------------------------------
>
>                 Key: CASSANDRA-3997
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3997
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Vijay
>            Assignee: Vijay
>            Priority: Minor
>              Labels: cache
>             Fix For: 1.2
>
>         Attachments: 0001-CASSANDRA-3997.patch, jna.zip
>
>
> Serializing cache uses native malloc and free by making FM pluggable, users will have
a choice of gcc malloc, TCMalloc or JEMalloc as needed. 
> Initial tests shows less fragmentation in JEMalloc but the only issue with it is that
(both TCMalloc and JEMalloc) are kind of single threaded (at-least they crash in my test otherwise).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message