From user-return-11859-apmail-hbase-user-archive=hbase.apache.org@hbase.apache.org Mon Aug 02 18:12:34 2010 Return-Path: Delivered-To: apmail-hbase-user-archive@www.apache.org Received: (qmail 67205 invoked from network); 2 Aug 2010 18:12:34 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 2 Aug 2010 18:12:34 -0000 Received: (qmail 73031 invoked by uid 500); 2 Aug 2010 18:12:33 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 72990 invoked by uid 500); 2 Aug 2010 18:12:32 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 72982 invoked by uid 99); 2 Aug 2010 18:12:32 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 18:12:32 +0000 X-ASF-Spam-Status: No, hits=2.9 required=10.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.212.41] (HELO mail-vw0-f41.google.com) (209.85.212.41) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 18:12:28 +0000 Received: by vws16 with SMTP id 16so2636628vws.14 for ; Mon, 02 Aug 2010 11:12:07 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.168.10 with SMTP id s10mr4505997vcy.50.1280772726895; Mon, 02 Aug 2010 11:12:06 -0700 (PDT) Received: by 10.220.190.70 with HTTP; Mon, 2 Aug 2010 11:12:06 -0700 (PDT) In-Reply-To: References: Date: Mon, 2 Aug 2010 11:12:06 -0700 Message-ID: Subject: Re: Which LZO library to use? From: Alex Kozlov To: user@hbase.apache.org Content-Type: multipart/alternative; boundary=0016364eecce6cb63b048cdb238e --0016364eecce6cb63b048cdb238e Content-Type: text/plain; charset=ISO-8859-1 The code is currently maintained by Kevin Weil and Todd Lipcon. For completeness, there is one more distribution at http://github.com/toddlipcon/hadoop-lzo. AFAIK, the Todd Lipcon's and Kevin Weil's distribution are synced. Most of the differences with google's code are bug fixes: the lzo file format itself had not changed and you can actually read the files created with lzop (the LZO command line tool). Their are no version compatibility issues currently. Alex K On Mon, Aug 2, 2010 at 3:25 AM, Eran Kutner wrote: > Hi, > I want to enable LZO compression on my cluster but see there are a few > alternatives and the wiki page itself is very confusing so it's not clear > what is the right choice. I was looking at this page: > http://wiki.apache.org/hadoop/UsingLzoCompression, at the top it > recommends > using Kevin Weil's version (which seems to be the same one released by > Twitter) but warns it doesn't contain all fixes and lower in the article it > refers to the original google code repository ( > http://code.google.com/p/hadoop-gpl-compression/). > The thing the concerns me most is future compatibility, so whichever > library > I pick now I want to be certain my data compressed will still be readable > when I I upgrade to the next major version of Hadoop and Hbase. It seems > that only the Google code project has newer releases compatible with future > version of Hadoop. > > So I'm looking for recommendations on which library to use. > > > Thanks, > Eran > --0016364eecce6cb63b048cdb238e--