mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 施伟伟 <shiweiwe...@gmail.com>
Subject Re: deadlock in FastByIDMap.find() ?
Date Thu, 10 Dec 2009 01:21:47 GMT
BTW, I am using iterator.remove() to remove elements from FastByIDMap &
FastIDSet.
Will this be a problem?

I have moved the filtering logic into normal Map and Set. Will test if this
problem persists after this change.

Weiwei

2009/12/6 Sean Owen <srowen@gmail.com>

> If he's using the normal class APIs, then shouldn't matter what the
> heck is being done to the class. I can't spot a problem that would
> fill up the hash table.
>
> I don't think he's messing with the implementation or data structures
> directly?
>
> But yes I am still quite puzzled, I haven't seen this, can't reproduce
> it even with some tests that guess at pathological patterns, and can't
> see anything wrong with the logic. I'm stumped on this one.
>
> On Sat, Dec 5, 2009 at 7:54 PM, Ted Dunning <ted.dunning@gmail.com> wrote:
> > Is it possible that weiwei is filling the fastMap explicitly?
> >
> > On Sat, Dec 5, 2009 at 1:31 AM, Sean Owen <srowen@gmail.com> wrote:
> >
> >> That all sounds fine, should be no problem with this usage pattern. I
> >> don't think the issue is where you think it is -- The problem seemed
> >> to be with the FastMap used to cache recommendations. Is there
> >> anything interesting about how this is used? I just added a couple
> >> unit tests for the cache to try to reproduce it but didn't see
> >> anything.
> >>
> >> You can remove CachingRecommender to see what happens but that would
> >> be a shame since it's just avoiding the problem. If there's an issue
> >> I'm keen to track it down.
> >>
> >> I can't figure out what the issue is. Maybe I can add some asserts to
> >> the code that could trigger some signs of a problem.
> >>
> >> 2009/12/5 施伟伟 <shiweiwei97@gmail.com>:
> >> > Thanks for reply.
> >> > It does not happen all the time. I saw this twice and could not easily
> >> > reproduce it.
> >> > Next time I will try testing it with jpda enabled.
> >> >
> >> > I am not sure if I am doing something wrong in the data model.
> >> > First I fill the FastMap with data read from external data source.
> >> > Then I iterate all entries in FastMap and find a set of items that I
> >> don't
> >> > need.
> >> > Finally I iterate the entries again to remove those items from
> FastMap.
> >> If
> >> > the item set of any userID becomes empty, the userID will be removed
> from
> >> > FastMap too.
> >> >
> >> > Maybe I should do these outside mahout using ordinary Map?
> >> >
> >> > Thanks,
> >> > Weiwei
> >> >
> >>
> >
> >
> >
> > --
> > Ted Dunning, CTO
> > DeepDyve
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message