From dev-return-1088-apmail-atlas-dev-archive=atlas.apache.org@atlas.incubator.apache.org Wed Sep 16 08:36:59 2015 Return-Path: X-Original-To: apmail-atlas-dev-archive@minotaur.apache.org Delivered-To: apmail-atlas-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E54D6179A5 for ; Wed, 16 Sep 2015 08:36:58 +0000 (UTC) Received: (qmail 60272 invoked by uid 500); 16 Sep 2015 08:36:58 -0000 Delivered-To: apmail-atlas-dev-archive@atlas.apache.org Received: (qmail 60229 invoked by uid 500); 16 Sep 2015 08:36:58 -0000 Mailing-List: contact dev-help@atlas.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@atlas.incubator.apache.org Delivered-To: mailing list dev@atlas.incubator.apache.org Received: (qmail 60213 invoked by uid 99); 16 Sep 2015 08:36:58 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 Sep 2015 08:36:58 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 6BBCB1A2074 for ; Wed, 16 Sep 2015 08:36:58 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.006 X-Spam-Level: X-Spam-Status: No, score=-0.006 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.006] autolearn=disabled Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id qoN68SGaDXFT for ; Wed, 16 Sep 2015 08:36:46 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with SMTP id 135672118F for ; Wed, 16 Sep 2015 08:36:46 +0000 (UTC) Received: (qmail 57467 invoked by uid 99); 16 Sep 2015 08:36:45 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 Sep 2015 08:36:45 +0000 Date: Wed, 16 Sep 2015 08:36:45 +0000 (UTC) From: "Shwetha G S (JIRA)" To: dev@atlas.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (ATLAS-58) Make hive hook reliable MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/ATLAS-58?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14747142#comment-14747142 ] Shwetha G S commented on ATLAS-58: ---------------------------------- Bug for making service/listener configurable using application.properties - ATLAS-103 > Make hive hook reliable > ----------------------- > > Key: ATLAS-58 > URL: https://issues.apache.org/jira/browse/ATLAS-58 > Project: Atlas > Issue Type: Sub-task > Reporter: Shwetha G S > Assignee: Shwetha G S > Labels: incompatible > Fix For: trunk > > Attachments: ATLAS-58-v2.patch, ATLAS-58.patch > > > Currently, hive hook executes in background thread pool and is an best effort approach to register entities. But this needs to be reliable for data governance to be effective > One way is - in hive hook, add the entities to some messaging framework and atlas server can read the entities from the message and register in atlas. Since, posting message is faster, we can do it synchronously and hence reliable entity registration. > We can start with kafka for messaging, but any other messaging framework should be pluggable -- This message was sent by Atlassian JIRA (v6.3.4#6332)