mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 施伟伟 <shiweiwe...@gmail.com>
Subject deadlock in FastByIDMap.find() ?
Date Fri, 04 Dec 2009 10:14:14 GMT
Hi,

I found my application based on mahout using 99% CPU after load testing.
There is no actual requests at the moment but the CPU usage kept high.
I generated several thread dump and found one thread may be get stuck an
endless loop in FastByIDMap.find().
I am not sure what is the problem is. I am using customized data model
implemented by myself. It is more or less the same as FileDataModel but
reading data from a different data source.
Please find the detailed info below.

Thanks,
Weiwei

  private int find(long key) {
    int theHashCode = (int) key & 0x7FFFFFFF; // make sure it's positive
    long[] keys = this.keys;
    int hashSize = keys.length;
    int jump = 1 + theHashCode % (hashSize - 2);
    int index = theHashCode % hashSize;
    long currentKey = keys[index];
    while (currentKey != NULL && (currentKey == REMOVED || key !=
currentKey)) {
      if (index < jump) {
        index += hashSize - jump;
      } else {
        index -= jump;
      }
      currentKey = keys[index];
    }
    return index;
  }

Below is the thread dump.

This is the thread that got the lock in
org.apache.mahout.cf.taste.impl.common.Cache.get
"http-10.90.39.156-23869-Processor24" daemon prio=1 tid=0x085bb618
nid=0x4f5f runnable [0x825a8000..0x825a8f30]
    at org.apache.mahout.cf.taste.impl.common.FastMap.find(FastMap.java:113)
    at org.apache.mahout.cf.taste.impl.common.FastMap.get(FastMap.java:123)
    at org.apache.mahout.cf.taste.impl.common.Cache.get(Cache.java:74)
    - locked <0x96eadf98> (a org.apache.mahout.cf.taste.impl.common.FastMap)
    at
org.apache.mahout.cf.taste.impl.recommender.CachingRecommender.recommend(CachingRecommender.java:118)


This is the thread that is blocked in
org.apache.mahout.cf.taste.impl.common.Cache.get. There are hundreds of such
threads in my thread dump file.
"http-10.90.39.156-23869-Processor272" daemon prio=1 tid=0x08405d78
nid=0x5479 waiting for monitor entry [0x731ef000..0x731f0130]
    at org.apache.mahout.cf.taste.impl.common.Cache.get(Cache.java:73)
    - waiting to lock <0x96eadf98> (a
org.apache.mahout.cf.taste.impl.common.FastMap)
    at
org.apache.mahout.cf.taste.impl.recommender.CachingRecommender.recommend(CachingRecommender.java:115)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message