jakarta-jcs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manfred Quasten (JIRA)" <j...@apache.org>
Subject [jira] Reopened: (JCS-52) remote diskcache, out of memory and lost objects using remoting
Date Mon, 15 Sep 2008 07:39:44 GMT

     [ https://issues.apache.org/jira/browse/JCS-52?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Manfred Quasten reopened JCS-52:
--------------------------------


The out of memory problem can be solved,  when I create a queue at client side.

jcs.auxiliary.RFailover=org.apache.jcs.auxiliary.remote.RemoteCacheFactory
jcs.auxiliary.RFailover.attributes=org.apache.jcs.auxiliary.remote.RemoteCacheAttributes
jcs.auxiliary.RFailover.attributes.FailoverServers=localhost:1102
jcs.auxiliary.RC.attributes.RemoveUponRemotePut=true
jcs.auxiliary.RFailover.attributes.GetOnly=false


jcs.region.myRegion1=RFailover
jcs.region.myRegion1.elementattributes.IsEternal=false
jcs.region.myRegion1.elementattributes.MaxLifeSeconds=60000
jcs.region.myRegion1.elementattributes.IsSpool=true
jcs.region.myRegion1.elementattributes.IsLateral=true
jcs.region.myRegion1.elementattributes.IsRemote=true
jcs.region.myRegion1.elementattributes.IsRemote=true





# DEFAULT CACHE REGION
jcs.default=RFailover
jcs.default.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
jcs.default.cacheattributes.MaxObjects=0
jcs.default.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache


# Indexed Disk Cache
###jcs.auxiliary.DC=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheFactory
###jcs.auxiliary.DC.attributes=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheAttributes
###jcs.auxiliary.DC.attributes.DiskPath=O:\\liantis/JCS/dump
###jcs.auxiliary.RFailover.attributes.DiskPath=dump

jcs.auxiliary.RFailover.attributes.OptimizeOnShutdown=true
jcs.auxiliary.RFailover.attributes.MaxRecycleBinSize=7500
jcs.auxiliary.RFailover.attributes.EventQueueType=POOLED

jcs.auxiliary.RFailover.attributes.EventQueuePoolName=disk_cache_event_queue



# Disk Cache pool
thread_pool.disk_cache_event_queue.boundarySize=50
thread_pool.disk_cache_event_queue.useBoundary=true
thread_pool.disk_cache_event_queue.maximumPoolSize=15
thread_pool.disk_cache_event_queue.minimumPoolSize=1
thread_pool.disk_cache_event_queue.keepAliveTime=3500000
thread_pool.disk_cache_event_queue.startUpSize=1

###thread_pool.disk_cache_event_queue.whenBlockedPolicy=RUN



#jcs.region.myRegion1=RFailover
#jcs.region.myRegion1.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
#jcs.region.myRegion1.cacheattributes.MaxObjects=100
#jcs.region.myRegion1.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache

I write 50 000 objects with 10000 characters and a simple key (the for-loop counter). After
this I  try to read the Keys and in a lot of cases 
the jcs.getFromGroup returns null.  The heapsize from client and server jvm are 64 M the java
default, because I can not give more Memory. We will use the  heapsize in our application,
that is the reason for us to evalute the JCS. 

My server-config is 

# Remote RMI Cache set up to failover
jcs.auxiliary.RFailover=org.apache.jcs.auxiliary.remote.RemoteCacheFactory
jcs.auxiliary.RFailover.attributes=org.apache.jcs.auxiliary.remote.RemoteCacheAttributes
#jcs.auxiliary.RFailover.attributes.FailoverServers=localhost:1102,localhost:1103
jcs.auxiliary.RC.attributes.RemoveUponRemotePut=true
jcs.auxiliary.RFailover.attributes.GetOnly=false

#jcs.region.testCache1=DC,RFailover
#jcs.region.testCache1.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
#jcs.region.testCache1.cacheattributes.MaxObjects=1000
#jcs.region.testCache1.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache

# Registry used to register and provide the
# IRemoteCacheService service.
registry.host=localhost
registry.port=1102
# call back port to local caches.
remote.cache.service.port=1102
# cluster setting
remote.cluster.LocalClusterConsistency=true
remote.cluster.AllowClusterGet=true

##############################################################
##### Default Region Configuration
jcs.default=DC
jcs.default.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
jcs.default.cacheattributes.MaxObjects=-1
jcs.default.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache
jcs.default.cacheattributes.UseMemoryShrinker=false
jcs.default.cacheattributes.ShrinkerIntervalSeconds=60

##############################################################
##### CACHE REGIONS
jcs.region.myRegion1=DC
jcs.region.myRegion1.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
jcs.region.myRegion1.cacheattributes.MaxObjects=-1
jcs.region.myRegion1.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache
#jcs.region.myRegion1.cacheattributes.DiskUsagePattern=SWAP
##############################################################
##### AUXILIARY CACHES
# Indexed Disk Cache
jcs.auxiliary.DC=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheFactory
jcs.auxiliary.DC.attributes=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheAttributes
###jcs.auxiliary.DC.attributes.DiskPath=O:\\liantis/JCS/dump
jcs.auxiliary.DC.attributes.DiskPath=dump
jcs.auxiliary.DC.attributes.MaxPurgatorySize=100
jcs.auxiliary.DC.attributes.MaxKeySize=-1
jcs.auxiliary.DC.attributes.OptimizeAtRemoveCount=300000
jcs.auxiliary.DC.attributes.OptimizeOnShutdown=true
jcs.auxiliary.DC.attributes.MaxRecycleBinSize=7500
jcs.auxiliary.DC.attributes.EventQueueType=POOLED
jcs.auxiliary.DC.attributes.EventQueuePoolName=disk_cache_event_queue


# Disk Cache pool
thread_pool.disk_cache_event_queue.boundarySize=50
thread_pool.disk_cache_event_queue.useBoundary=true
thread_pool.disk_cache_event_queue.maximumPoolSize=15
thread_pool.disk_cache_event_queue.minimumPoolSize=1
thread_pool.disk_cache_event_queue.keepAliveTime=350000
thread_pool.disk_cache_event_queue.startUpSize=1

I think, it is only a smal thing in my queue configuration at server side (or better I hope
so) 

I've not said that this is a bug, it is a question , but an imprortant question,because it
blocks my work . 

This is the status report from the Servlet, after the put from client is finiseh:

---------------------------
Region Name = myRegion1
HitCountRam = 0
HitCountAux = 2306
---------------------------LRU Memory Cache
List Size = 0
Map Size = 0
Put Count = 50000
Hit Count = 0
Miss Count = 2927
---------------------------Indexed Disk Cache
Is Alive = true
Key Map Size = 39826
Data File Length = 1508119657
Hit Count = 2306
Bytes Free = 37866
Optimize Operation Count = 6052
Times Optimized = 0
Recycle Count = 6051
Recycle Bin Size = 1
Startup Size = 0
Purgatory Hits = 0
Purgatory Size = 0
Working = true
Destroyed = false
Empty = true
Queue Size = 0
Queue Capacity = 50
Pool Size = 11
Maximum Pool Size = 15

IN LRU Memory Cache the put cout is 50000, but the  Key Map Size = 39826. So I think the data
are lost on the serverside. I belive something
in the configuration of my serverside queue is wrong.   To make it more simple, I have running
client and server at the same pc and in the same vm.


It is not easy to use jcs, because the configuration is complicated. 



> remote diskcache, out of memory and lost objects using remoting 
> ----------------------------------------------------------------
>
>                 Key: JCS-52
>                 URL: https://issues.apache.org/jira/browse/JCS-52
>             Project: JCS
>          Issue Type: Question
>          Components: RMI Remote Cache
>    Affects Versions: jcs-1.3
>         Environment: java6, xp, tomcat6 
>            Reporter: Manfred Quasten
>            Assignee: Aaron Smuts
>            Priority: Blocker
>             Fix For: jcs-1.3
>
>
> I try to put  and get  objects from a remote cache, that is running on Tomcat.  I use
index_disk_cache on the server.  The communication between client and server works. In case
of large objects or a large number of objects I get a java.lang.OutOfMemoryError 
> Exception in thread "CacheEventQueue.QProcessor-myRegion1" java.lang.OutOfMemoryError:
Java heap space
> 	at sun.reflect.ByteVectorImpl.trim(ByteVectorImpl.java:52)
> 	at sun.reflect.MethodAccessorGenerator.generate(MethodAccessorGenerator.java:370)
> 	at sun.reflect.MethodAccessorGenerator.generateSerializationConstructor(MethodAccessorGenerator.java:95)
> 	at sun.reflect.ReflectionFactory.newConstructorForSerialization(ReflectionFactory.java:313)
> 	at java.io.ObjectStreamClass.getSerializableConstructor(ObjectStreamClass.java:1327)
> 	at java.io.ObjectStreamClass.access$1500(ObjectStreamClass.java:52)
> 	at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:437)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:413)
> 	at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:310)
> 	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1106)
> 	at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509)
> 	at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1474)
> 	at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
> 	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
> 	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)
> 	at sun.rmi.server.UnicastRef.marshalValue(UnicastRef.java:274)
> 	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:133)
> 	at org.apache.jcs.auxiliary.remote.server.RemoteCacheServer_Stub.update(Unknown Source)
> 	at org.apache.jcs.auxiliary.remote.RemoteCache.update(RemoteCache.java:210)
> 	at org.apache.jcs.engine.CacheAdaptor.handlePut(CacheAdaptor.java:91)
> 	at org.apache.jcs.engine.CacheEventQueue$PutEvent.doRun(CacheEventQueue.java:688)
> 	at org.apache.jcs.engine.CacheEventQueue$AbstractCacheEvent.run(CacheEventQueue.java:607)
> 	at org.apache.jcs.engine.CacheEventQueue$QProcessor.run(CacheEventQueue.java:575)
> My configurations are: 
> 1) client:
> jcs.auxiliary.RFailover=org.apache.jcs.auxiliary.remote.RemoteCacheFactory
> jcs.auxiliary.RFailover.attributes=org.apache.jcs.auxiliary.remote.RemoteCacheAttributes
> jcs.auxiliary.RFailover.attributes.FailoverServers=pchl:1102
> jcs.auxiliary.RC.attributes.RemoveUponRemotePut=true
> jcs.auxiliary.RFailover.attributes.GetOnly=false
> jcs.auxiliary.RFailover.attributes.MaxObjects=100
> jcs.region.myRegion1=RFailover
> #jcs.region.myRegion1=RFailover
> #jcs.region.myRegion1.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
> #jcs.region.myRegion1.cacheattributes.MaxObjects=100
> #jcs.region.myRegion1.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache
> Server:
> # Remote RMI Cache set up to failover
> jcs.auxiliary.RFailover=org.apache.jcs.auxiliary.remote.RemoteCacheFactory
> jcs.auxiliary.RFailover.attributes=org.apache.jcs.auxiliary.remote.RemoteCacheAttributes
> jcs.auxiliary.RC.attributes.RemoveUponRemotePut=true
> jcs.auxiliary.RFailover.attributes.GetOnly=false
> # this didn't help, the memory error occurs with and without
> #jcs.region.testCache1=DC,RFailover
> #jcs.region.testCache1.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
> #jcs.region.testCache1.cacheattributes.MaxObjects=1000
> #jcs.region.testCache1.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache
> # Registry used to register and provide the
> # IRemoteCacheService service.
> registry.host=localhost
> registry.port=1102
> # call back port to local caches.
> remote.cache.service.port=1102
> # cluster setting
> remote.cluster.LocalClusterConsistency=true
> remote.cluster.AllowClusterGet=true
> ##############################################################
> ##### Default Region Configuration
> jcs.default=DC
> jcs.default.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
> jcs.default.cacheattributes.MaxObjects=100
> jcs.default.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache
> jcs.default.cacheattributes.UseMemoryShrinker=false
> jcs.default.cacheattributes.ShrinkerIntervalSeconds=60
> ##############################################################
> ##### CACHE REGIONS
> jcs.region.myRegion1=DC
> jcs.region.myRegion1.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
> jcs.region.myRegion1.cacheattributes.MaxObjects=-1
> jcs.region.myRegion1.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache
> #jcs.region.myRegion1.cacheattributes.DiskUsagePattern=SWAP
> ##############################################################
> ##### AUXILIARY CACHES
> # Indexed Disk Cache
> jcs.auxiliary.DC=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheFactory
> jcs.auxiliary.DC.attributes=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheAttributes
> ###jcs.auxiliary.DC.attributes.DiskPath=O:\\liantis/JCS/dump
> jcs.auxiliary.DC.attributes.DiskPath=dump
> jcs.auxiliary.DC.attributes.MaxPurgatorySize=100
> jcs.auxiliary.DC.attributes.MaxKeySize=-1
> jcs.auxiliary.DC.attributes.OptimizeAtRemoveCount=300000
> jcs.auxiliary.DC.attributes.OptimizeOnShutdown=true
> jcs.auxiliary.DC.attributes.MaxRecycleBinSize=7500
> jcs.auxiliary.DC.attributes.EventQueueType=POOLED
> jcs.auxiliary.DC.attributes.EventQueuePoolName=disk_cache_event_queue
> # Disk Cache pool
> thread_pool.disk_cache_event_queue.boundarySize=50
> thread_pool.disk_cache_event_queue.useBoundary=true
> thread_pool.disk_cache_event_queue.maximumPoolSize=15
> thread_pool.disk_cache_event_queue.minimumPoolSize=1
> thread_pool.disk_cache_event_queue.keepAliveTime=3500
> thread_pool.disk_cache_event_queue.startUpSize=1
> pchl is the server on that Tomcat is running. 
> In case of smal objects I have the problem, that I will  lose  objects again. I've configured
the queue and the threadpool at the server,  but it seams, that there is something too, that
I've forgotten... 
> I'll work with a absolute minimum on java heapsize,  that is the reson to use a cache
in our case.
> The problems will occur with and without the outcommented lines in the clientconfiguration

> The reason for the questions during the last days are , that we evaluate caching architectures
and performance using jcs ...

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-dev-help@jakarta.apache.org


Mime
View raw message