hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei-Chiu Chuang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15566) Support Opentracing
Date Thu, 02 May 2019 00:45:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16831360#comment-16831360
] 

Wei-Chiu Chuang commented on HADOOP-15566:
------------------------------------------

I spent past few months thinking about the ways to add opentracing.

[~carlosalberto] and my PoC are ok (for the record, here's what [~fabbri], [~rizaon] and I
did before https://github.com/jojochuang/hadoop-common/commits/hac-271), but the problems
are (1) the work were not backward compatible (2) the change were too much, hard to review.

[~smeng] is working on a patch to approach this, with minimal code change as possible. I will
post a design doc soon.

Recently, while I worked on KMS-o-meter tool, a workload replay tool for KMS, I added opentracing
in it and in KMS server/client.(courtesy of [~elek]'s tracing work in ozone, I simply took
part of it). The ability to understand system performance in such details is unprecedented.
We were able to fix a system scalability  bug, verify it in record time. Will file a jira
and contribute the opentracing code in KMS server/client back to the community.

> Support Opentracing
> -------------------
>
>                 Key: HADOOP-15566
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15566
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: metrics
>    Affects Versions: 3.1.0
>            Reporter: Todd Lipcon
>            Priority: Major
>              Labels: security
>         Attachments: Screen Shot 2018-06-29 at 11.59.16 AM.png, ss-trace-s3a.png
>
>
> The HTrace incubator project has voted to retire itself and won't be making further releases.
The Hadoop project currently has various hooks with HTrace. It seems in some cases (eg HDFS-13702)
these hooks have had measurable performance overhead. Given these two factors, I think we
should consider removing the HTrace integration. If there is someone willing to do the work,
replacing it with OpenTracing might be a better choice since there is an active community.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message