jakarta-jcs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Travis Savo <ts...@IFILM.com>
Subject RE: outstanding issues?
Date Wed, 14 Apr 2004 23:15:56 GMT
I wasn't trying to say the design of JCS is complex; The design of the
current LRU Memory Cache is needlessly complex: it attempts to implement a
Hash Map backed by a Linked List for ordering from scratch, which has proven
to be buggy and is anything but easy to read and maintain. Eliminating this
custom implementation eliminates a great deal of complexity from the
LRUMemoryCache, making the code -much- easier to read and maintain.

The new LRU memory cache is simple, faster, and implements all the
functionality of the existing LRU memory cache. Were it otherwise I would
completely agree with you. But my argument remains: why have two LRU memory
stores with identical functionality, except one leaks memory and is slower,
while the other is proven to be more stable? Deprecating one doesn't do
anything to help the user be successful with JCS because it's not a publicly
exposed interface, so 99% of the users won't realize it's deprecated unless
you go to great lengths to make it obvious that they should be using one
over the other. And why are we offering them choices of leaky, hard to
maintain, and slower vs. stable, well tested, and faster? I understand the
reason for the pluggable architecture, and 'giving the user enough rope to
hang themselves with', but why supply options that shouldn't be used under
any circumstances when a perfectly acceptable altertinative is available?
It's not a 'new auxiliary cache'! It's a faster, less bug prone
implementation of an existing one.

I'm 100% for the pluggable architecture because it allows me an entry point
for implementing my own custom storage mechanisms for my custom
implementation. But we're talking about the basic underlying memory store
which is the default for most of the configurations you'll come across. Why
would we supply potentially bad choices at that level?

The new LRU Memory Cache is significantly different in implementation, but
identical in purpose, design, and features as the existing LRU Memory Cache.

The proposed fix for the lateral, as I've covered in my prior posts, is as
follows:

diff -r1.5 LateralTCPSender.java
228c228,231
<         oos.writeObject(led);
---
>       	if (socket.getInputStream().available() > 0) {
>       		socket.getInputStream().read(new
byte[socket.getInputStream().available()]);
>       	}
>       	oos.writeObject(led);

It remains untested, but the idea is: If we have something in the
InputStream before we send the request, we need to flush the input stream.


As for the changes to the disk cache: It's essentially identical in design;
You serialize the element and store it in the first available block of free
space. The major changes are encapsulating the semantics of disk storage in
the IndexedDisk, rather than the IndexdDiskCache. The IndexedDiskCache now
only encapsulates the read/write locking semantics and doesn't deal with the
creation of IndexedDiskElementDescriptors, or their position within in the
IndexedDisk, greatly simplifying the implementation, and allowing for other
implementations of IndexedDisk to be plugged in without any changes to
IndexedDiskCache.

-Travis Savo



-----Original Message-----
From: Aaron Smuts [mailto:aasmuts@wisc.edu]
Sent: Wednesday, April 14, 2004 2:38 PM
To: 'Turbine JCS Developers List'
Subject: RE: outstanding issues?


The design of JCS is not complex.  It does try to support most of the
JCACHE API.  

There is a problem using something with less features, at lest if the
more featurific version is not available.

If you have a simpler, faster alternative then it should be another
auxiliary.  If it proves to have all the same functionality, then the
old auxiliary can be depreciated.  I'm trying to figure out what kind of
situation we are in.

You're memory cache is significantly different.  I need to go over it in
more detail.  I'd like to fix any problems with the existing LRU if
possbile also.

If you have a simple fix to the lateral, can you send the changes in an
email.  I'd love to improve it.

The current disk cache only uses one file, except on shutdown.  I don't
see any advantage by not having an extra key file.  I have to look into
your disk cache.  What exactly are the changes?

Aaron


> -----Original Message-----
> From: Travis Savo [mailto:tsavo@IFILM.com]
> Sent: Wednesday, April 14, 2004 2:20 PM
> To: 'Turbine JCS Developers List'
> Subject: RE: outstanding issues?
> 
> Definitely check out the EHCache tests. They have several tests which
> expose
> the flaws.
> 
> The disk locking problem happens only under very heavy load. The
> implementation taken from EHCache doesn't exhibit the same problems,
only
> requires one file per region, and can be easily and reliably be made
> persistable across restarts. Is there something wrong with using it?
> 
> As covered in my prior posts with patches, the problem with lateral
> distribution is the LateralTCPService can get a socket stream with
dirty
> data from prior requests, leading to mismatched objects returned on
> lateral
> get. My patch demonstrates an obvious place to flush the buffer.
> 
> The memory leak in memory cache happens when the linked list becomes
> inconsistent with the map. EHCache tests to expose this suggest the
> problem
> is with removeAll, but I've seen it with just get/put/remove. The LRU
> memory
> store implementation from EHCache uses an java.util.LinkedHashMap, or
if
> it's not available, a org.apache.commons.collections.map.LRUMap,
thereby
> dramatically reducing the complexity of the code, and eliminating
> opportunities for the linked list and map to get out of sync.
> 
> It's also 79% faster for the 'insert 5 million typical CacheElements,
then
> get each of them by key' test, and 293% faster for the 'insert 5
million
> typical CacheElements and get each of them by key, then remove them
one at
> a
> time by key' (stastics borrowed from
>
http://ehcache.sourceforge.net/documentation/index.html#physicalstores).
> Again, is there something wrong with using it? Why go to all the work
of
> fixing a complex design when a simpler, faster one exists, and is
working
> without problems?
> 
> The other issues I'm aware of is with RemoteCache, where removes will
> generate exponential remove requests. I'm working on a patch for it
from
> my
> code base, as well as the change from byte to integer for cache ID's,
> which
> will cause problems with 2 or more remote caches under flakey network
> conditions.
> 
> -Travis Savo
> 
> 
> 
> -----Original Message-----
> From: Aaron Smuts [mailto:aasmuts@wisc.edu]
> Sent: Wednesday, April 14, 2004 11:41 AM
> To: 'Turbine JCS Developers List'
> Subject: outstanding issues?
> 
> 
> I want to knock out any remaining issues.
> 
> Can someone let me know if they can get the disk cache to lock.  I
can't
> make it happen.
> 
> Can someone explain the purported problem with the lateral
distribution.
> 
> Was the "memory leak" in the memory cache just the problem with
expired
> elements not getting removed.  If so, it is solve.  If not, can
someone
> explain it to me.
> 
> If you are aware of any other issues, please let me know and I'll work
> on them.
> 
> Thanks,
> 
> Aaron
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: turbine-jcs-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
turbine-jcs-dev-help@jakarta.apache.org
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: turbine-jcs-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
turbine-jcs-dev-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: turbine-jcs-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: turbine-jcs-dev-help@jakarta.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: turbine-jcs-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: turbine-jcs-dev-help@jakarta.apache.org


Mime
View raw message