sentry-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <>
Subject [jira] [Updated] (SENTRY-664) After Namenode is restarted, Path updates remain unsynched
Date Wed, 04 Mar 2015 05:27:04 GMT


Arun Suresh updated SENTRY-664:
    Attachment: SENTRY-664.1.patch

Attaching trivial patch.

I have also included some log message modifications that helped me debug the issue..

The fix is pretty trivial (It was caused due the fact that the lastSentSeqNum in the MetaStorePlugin
was initialized to -1, while it should have been equal to the current value of seqNum). Unfortunately
writing a testcase is hard.. more so because the "miniDfs" tools used in the testCase has
a bug in the version of hadoop sentry depends on. I have put in the additional tests but have
commented them out and have placed a TODO to remove the comment once we have a working miniDFS

> After Namenode is restarted, Path updates remain unsynched
> ----------------------------------------------------------
>                 Key: SENTRY-664
>                 URL:
>             Project: Sentry
>          Issue Type: Bug
>    Affects Versions: 1.4.0
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>              Labels: sentry-hdfs
>         Attachments: SENTRY-664.1.patch
> Path updates remain un-synced after a Namenode restart.
> Ideally, after a restart, the Namenode receives both the Path and Permission updates
from sentry service. These updates are ordered by a sequence Id. Also the first Path/Permission
update is supposed to be a FULL update. 
> From observing the logs, It looks like after namenode restart, the sequence of PathsUpdate
objects do not start with a FULL update (The PermissionsUpdates are fine though). Thus the
namenode remains unsynched with respect to path information.
> A workaround is to restart the metastore after the namenode has restarted. This will
ensure HMS will send a FULL update to Sentry. This will cause Sentry service will flush its
existing update queue, so the next time the namenode asks for an update, this FULL update
will be sent. 

This message was sent by Atlassian JIRA

View raw message