From mahout-user-return-1863-apmail-lucene-mahout-user-archive=lucene.apache.org@lucene.apache.org Thu Dec 03 20:29:49 2009 Return-Path: Delivered-To: apmail-lucene-mahout-user-archive@minotaur.apache.org Received: (qmail 7667 invoked from network); 3 Dec 2009 20:29:49 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 3 Dec 2009 20:29:49 -0000 Received: (qmail 76543 invoked by uid 500); 3 Dec 2009 20:29:48 -0000 Delivered-To: apmail-lucene-mahout-user-archive@lucene.apache.org Received: (qmail 76449 invoked by uid 500); 3 Dec 2009 20:29:46 -0000 Mailing-List: contact mahout-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mahout-user@lucene.apache.org Delivered-To: mailing list mahout-user@lucene.apache.org Received: (qmail 76437 invoked by uid 99); 3 Dec 2009 20:29:46 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Dec 2009 20:29:46 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of ted.dunning@gmail.com designates 209.85.160.46 as permitted sender) Received: from [209.85.160.46] (HELO mail-pw0-f46.google.com) (209.85.160.46) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Dec 2009 20:29:37 +0000 Received: by pwj17 with SMTP id 17so1631357pwj.5 for ; Thu, 03 Dec 2009 12:29:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :from:date:message-id:subject:to:content-type; bh=/uyMLB13HjyWNkiPf0YLiCYMry2VIhb620pFwZVJmcs=; b=eyVRXWog1vnSwQq/hPxxlnGXDm65OOX/1G8opt6gIBuut6PiSoDfUwvbhCfn1yvjQA IzIR5fPiRH7Olb+2Or8pV4fhQj9SEOVqC5++ky7aPQlJDXdFjYmKJ6e9ShoX6NUYHRFp vtagwtEj4Ub+0oEP3JmcMGW2T2xrDlxHJNtCU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=c6rqcnnw2vgDMVKmStwyiHLeO5vPy0SpP6QGuq722gsUvwR8DJPiujVLBYJb870p7Z 3S7GpWZqEPtX8GCGBLVZxcPlTNY8vq07aFw9ZRyBcKNgbJghbowOc1TpVQ2lj26S1q8t GzEZHsHGf0Yb82UN3zA0ifrbYJdqX9mQjArUQ= MIME-Version: 1.0 Received: by 10.115.102.38 with SMTP id e38mr2688380wam.207.1259872155062; Thu, 03 Dec 2009 12:29:15 -0800 (PST) In-Reply-To: <4b124c310912031055q16acd8cdrb9e13f85bc4f1d27@mail.gmail.com> References: <989f35a80909070603h57867000s78f34c83ae6605c5@mail.gmail.com> <989f35a80909091542p3c882630yd97c38a84712f84@mail.gmail.com> <4b124c310912031031k50c3e253hc955291c7d9c2b78@mail.gmail.com> <4b124c310912031055q16acd8cdrb9e13f85bc4f1d27@mail.gmail.com> From: Ted Dunning Date: Thu, 3 Dec 2009 12:28:55 -0800 Message-ID: Subject: Re: Taste-GenericItemBasedRecommender To: mahout-user@lucene.apache.org Content-Type: multipart/alternative; boundary=0016e64c3de843d42a0479d8d80a X-Virus-Checked: Checked by ClamAV on apache.org --0016e64c3de843d42a0479d8d80a Content-Type: text/plain; charset=UTF-8 I think you can merge my passes into a single pass in which you compute the row and column sums at the same time that you compute the product. That is more complicated, though, and I hate fancy code. So you are right in practice that I have always had two passes. (although pig might be clever enough by now to merge them) There is another pass in which you use all of the sums to do the sparsification. I don't know if that could be done in the same pass or not. On Thu, Dec 3, 2009 at 10:55 AM, Jake Mannix wrote: > But yeah, in the case at hand, maybe that transposition is a bunch of work. > But is it more work to do my two MR jobs than to do your two MR jobs? Both > look > like two full passes over the data, right? > -- Ted Dunning, CTO DeepDyve --0016e64c3de843d42a0479d8d80a--