From dev-return-5106-apmail-tajo-dev-archive=tajo.apache.org@tajo.incubator.apache.org Tue Feb 25 01:20:08 2014 Return-Path: X-Original-To: apmail-tajo-dev-archive@minotaur.apache.org Delivered-To: apmail-tajo-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 64B521004A for ; Tue, 25 Feb 2014 01:20:08 +0000 (UTC) Received: (qmail 16488 invoked by uid 500); 25 Feb 2014 01:20:07 -0000 Delivered-To: apmail-tajo-dev-archive@tajo.apache.org Received: (qmail 16438 invoked by uid 500); 25 Feb 2014 01:20:07 -0000 Mailing-List: contact dev-help@tajo.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@tajo.incubator.apache.org Delivered-To: mailing list dev@tajo.incubator.apache.org Received: (qmail 16428 invoked by uid 99); 25 Feb 2014 01:20:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 25 Feb 2014 01:20:07 +0000 X-ASF-Spam-Status: No, hits=-1998.3 required=5.0 tests=ALL_TRUSTED,HTML_MESSAGE,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Tue, 25 Feb 2014 01:20:04 +0000 Received: (qmail 15288 invoked by uid 99); 25 Feb 2014 01:19:42 -0000 Received: from minotaur.apache.org (HELO minotaur.apache.org) (140.211.11.9) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 25 Feb 2014 01:19:42 +0000 Received: from localhost (HELO mail-qg0-f51.google.com) (127.0.0.1) (smtp-auth username hyunsik, mechanism plain) by minotaur.apache.org (qpsmtpd/0.29) with ESMTP; Tue, 25 Feb 2014 01:19:42 +0000 Received: by mail-qg0-f51.google.com with SMTP id q108so17133569qgd.10 for ; Mon, 24 Feb 2014 17:19:41 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=mime-version:date:message-id:subject:from:to:content-type; bh=frRuQ/KN9tF4nTyWKugoMQSCNjuvrF3vpqlPbW0IqPY=; b=lHDZL5h4K9689MDVY98GmsYZYMQr6VhxzId0P8n4Q1nN3mRvDqTWs09Au0FGC3ufBD N+oOd9Gy2LlMPZV/PduyZAR24KBMekiwrFwXTxfFowaZ6S6JUt99jOGgcJvr8tmLgGqC ya3nldUM/SWiIGBCFA1TN6yWdABgBQgRxFZqoVxLJNd/yM/7JtrxXn14tOFKJMHocsYh 9bKx0vihXnpmmNbqzANUC5rhYWUuQHIMWQuXUfqke2gEXxLZa6eXu6GESJyBb2FL0/n6 Ihe6fenJC3CUeGbgqDVO9FtXgYJ1Va8x01GLyXluvl6U57qNOpO3Q2btVj4+oah2+AEG UbLA== MIME-Version: 1.0 X-Received: by 10.224.7.69 with SMTP id c5mr34268783qac.58.1393291181414; Mon, 24 Feb 2014 17:19:41 -0800 (PST) Received: by 10.96.74.100 with HTTP; Mon, 24 Feb 2014 17:19:41 -0800 (PST) Date: Tue, 25 Feb 2014 10:19:41 +0900 Message-ID: Subject: [FYI] some remarkable features in hadoop 2.3.0 From: Hyunsik Choi To: tajo-dev Content-Type: multipart/alternative; boundary=001a11c24edaefe18e04f330e111 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c24edaefe18e04f330e111 Content-Type: text/plain; charset=ISO-8859-1 Hi folks, As you already know, Hadoop 2.3.0 release. While I'm reading the changes, I noted some new features that Tajo should consider. Centralized cache management in HDFS - https://issues.apache.org/jira/browse/HDFS-4949 Ealier, Min mentioned cached table. In offline, I discussed HDFS-4949 with him. It may be a candidate feature for our goal. Enable support for heterogeneous storages in HDFS - DN as a collection of storages - https://issues.apache.org/jira/browse/HDFS-2832 It's for different storage medias like SSD and HDD. Add a directbuffer Decompressor API to hadoop - https://issues.apache.org/jira/browse/HADOOP-10047 We already use compression/decompression in text file. We also should adopt comp/decomp to other file formats. For that, HDFS-10047 may be a nice candidate feature to be used. - hyunsik --001a11c24edaefe18e04f330e111--