hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-11029) LocalFS Statistics performs thread local call per byte written
Date Fri, 29 Aug 2014 20:49:53 GMT
Gopal V created HADOOP-11029:

             Summary: LocalFS Statistics performs thread local call per byte written
                 Key: HADOOP-11029
                 URL: https://issues.apache.org/jira/browse/HADOOP-11029
             Project: Hadoop Common
          Issue Type: Bug
            Reporter: Gopal V
         Attachments: local-fs-locking.png

This code is there in the hot-path of IFile writer via RawLocalFileSystem.


>From a preliminary glance, the lock prefix calls are coming from a threadlocal.get() within

     * Get or create the thread-local data associated with the current thread.
    private StatisticsData getThreadData() {
      StatisticsData data = threadData.get();
      if (data == null) {
        data = new StatisticsData(
            new WeakReference<Thread>(Thread.currentThread()));
        synchronized(this) {
          if (allData == null) {
            allData = new LinkedList<StatisticsData>();
      return data;

     * Increment the bytes read in the statistics
     * @param newBytes the additional bytes read
    public void incrementBytesRead(long newBytes) {
      getThreadData().bytesRead += newBytes;

This is incredibly inefficient when used from FSDataOutputStream

    public void write(int b) throws IOException {
      if (statistics != null) {

This message was sent by Atlassian JIRA

View raw message