tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rajesh Balamohan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (TEZ-3720) Optimize SimpleHistoryLoggingService: FileSystem$Stats updates are expensive than writes
Date Thu, 11 May 2017 00:41:05 GMT
Rajesh Balamohan created TEZ-3720:
-------------------------------------

             Summary: Optimize SimpleHistoryLoggingService: FileSystem$Stats updates are expensive
than writes
                 Key: TEZ-3720
                 URL: https://issues.apache.org/jira/browse/TEZ-3720
             Project: Apache Tez
          Issue Type: Bug
            Reporter: Rajesh Balamohan


In some cases, Time taken to update FS$Stats is more than outputstream.write. E.g In SimpleHistoryLoggingService,
we can easily reduce the number of FS$Stats calls by half.

I haven't checked with ATS15 (in yarn side) where JSONGenerator ends up using FSDataOutputStream
and timeline entities are written. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message