atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (Jira)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-3519) pool performance in hive lineage query
Date Thu, 21 Nov 2019 19:45:00 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16979570#comment-16979570
] 

ASF subversion and git services commented on ATLAS-3519:
--------------------------------------------------------

Commit bdd110dfe7481815d3a19ebc6acb4fdefacc93ed in atlas's branch refs/heads/master from leslizhang
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=bdd110d ]

ATLAS-3519: reuse script engine to improve lineage query performance

Signed-off-by: Madhan Neethiraj <madhan@apache.org>
Signed-off-by: Sarath Subramanian <sarath@apache.org>


> pool performance in hive lineage query 
> ---------------------------------------
>
>                 Key: ATLAS-3519
>                 URL: https://issues.apache.org/jira/browse/ATLAS-3519
>             Project: Atlas
>          Issue Type: Improvement
>          Components:  atlas-core
>    Affects Versions: 1.1.0, 2.0.0
>            Reporter: zhangxiong
>            Assignee: zhangxiong
>            Priority: Minor
>              Labels: lineage, performance
>             Fix For: 2.1.0
>
>         Attachments: ATLAS-3519.patch
>
>          Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> when query lineage info for hive, the throughput reachs only 3 query per second.
> we use jstack and jfr+flame graph tools and found that the  GremlinGroovyScriptEngine
object loading class  logic takes much of the time. so we make the GremlinGroovyScriptEngine
a global singleton object, which would be shared accross all query threads. finally the query
performance improved much and got 200 qps.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message