hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From László Bodor (Jira) <j...@apache.org>
Subject [jira] [Updated] (HIVE-24085) Optimize build times of druid related modules
Date Thu, 27 Aug 2020 16:08:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-24085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

László Bodor updated HIVE-24085:
--------------------------------
    Description: 
IMO druid modules relatively take long compared to others, I mean these below:
{code}
[INFO] Hive Druid Handler 4.0.0-SNAPSHOT .................. SUCCESS [ 19.663 s]
[INFO] Hive Integration - QFile Druid Tests 4.0.0-SNAPSHOT  SUCCESS [ 37.034 s]
{code}

I accept a longer time only from core fundamental modules (e.g. hive-exec, metastore).

{code}
 mvn clean install -DskipTests -Pitests
{code}

{code}
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Hive Storage API 2.7.0-SNAPSHOT .................... SUCCESS [  7.520 s]
[INFO] Hive 4.0.0-SNAPSHOT ................................ SUCCESS [  2.230 s]
[INFO] Hive Classifications 4.0.0-SNAPSHOT ................ SUCCESS [  0.616 s]
[INFO] Hive Shims Common 4.0.0-SNAPSHOT ................... SUCCESS [  3.686 s]
[INFO] Hive Shims 0.23 4.0.0-SNAPSHOT ..................... SUCCESS [  6.012 s]
[INFO] Hive Shims Scheduler 4.0.0-SNAPSHOT ................ SUCCESS [  2.360 s]
[INFO] Hive Shims 4.0.0-SNAPSHOT .......................... SUCCESS [  1.806 s]
[INFO] Hive Standalone Metastore 4.0.0-SNAPSHOT ........... SUCCESS [  1.489 s]
[INFO] Hive Standalone Metastore Common Code 4.0.0-SNAPSHOT SUCCESS [ 26.412 s]
[INFO] Hive Common 4.0.0-SNAPSHOT ......................... SUCCESS [  7.406 s]
[INFO] Hive Service RPC 4.0.0-SNAPSHOT .................... SUCCESS [  3.394 s]
[INFO] Hive Serde 4.0.0-SNAPSHOT .......................... SUCCESS [  6.063 s]
[INFO] Hive Metastore 4.0.0-SNAPSHOT ...................... SUCCESS [  3.430 s]
[INFO] Hive Vector-Code-Gen Utilities 4.0.0-SNAPSHOT ...... SUCCESS [  0.574 s]
[INFO] Hive Parser 4.0.0-SNAPSHOT ......................... SUCCESS [ 12.159 s]
[INFO] Hive UDF 4.0.0-SNAPSHOT ............................ SUCCESS [  1.250 s]
[INFO] Hive Llap Common 4.0.0-SNAPSHOT .................... SUCCESS [  3.412 s]
[INFO] Hive Llap Client 4.0.0-SNAPSHOT .................... SUCCESS [  1.882 s]
[INFO] Hive Llap Tez 4.0.0-SNAPSHOT ....................... SUCCESS [  2.888 s]
[INFO] Hive Spark Remote Client 4.0.0-SNAPSHOT ............ SUCCESS [  3.946 s]
[INFO] Hive Metastore Server 4.0.0-SNAPSHOT ............... SUCCESS [ 20.209 s]
[INFO] Hive Query Language 4.0.0-SNAPSHOT ................. SUCCESS [01:06 min]
[INFO] Hive Llap Server 4.0.0-SNAPSHOT .................... SUCCESS [  9.777 s]
[INFO] Hive Service 4.0.0-SNAPSHOT ........................ SUCCESS [  8.090 s]
[INFO] Hive Accumulo Handler 4.0.0-SNAPSHOT ............... SUCCESS [  7.768 s]
[INFO] Hive JDBC 4.0.0-SNAPSHOT ........................... SUCCESS [ 24.253 s]
[INFO] Hive Beeline 4.0.0-SNAPSHOT ........................ SUCCESS [  4.741 s]
[INFO] Hive CLI 4.0.0-SNAPSHOT ............................ SUCCESS [  2.852 s]
[INFO] Hive Contrib 4.0.0-SNAPSHOT ........................ SUCCESS [  1.648 s]
[INFO] Hive Druid Handler 4.0.0-SNAPSHOT .................. SUCCESS [ 19.663 s]
[INFO] Hive HBase Handler 4.0.0-SNAPSHOT .................. SUCCESS [  5.218 s]
[INFO] Hive JDBC Handler 4.0.0-SNAPSHOT ................... SUCCESS [  2.004 s]
[INFO] Hive HCatalog 4.0.0-SNAPSHOT ....................... SUCCESS [  0.764 s]
[INFO] Hive HCatalog Core 4.0.0-SNAPSHOT .................. SUCCESS [  4.746 s]
[INFO] Hive HCatalog Pig Adapter 4.0.0-SNAPSHOT ........... SUCCESS [  2.959 s]
[INFO] Hive HCatalog Server Extensions 4.0.0-SNAPSHOT ..... SUCCESS [  2.381 s]
[INFO] Hive HCatalog Webhcat Java Client 4.0.0-SNAPSHOT ... SUCCESS [  2.970 s]
[INFO] Hive HCatalog Webhcat 4.0.0-SNAPSHOT ............... SUCCESS [ 12.019 s]
[INFO] Hive HPL/SQL 4.0.0-SNAPSHOT ........................ SUCCESS [  4.274 s]
[INFO] Hive Streaming 4.0.0-SNAPSHOT ...................... SUCCESS [  2.357 s]
[INFO] Hive Llap External Client 4.0.0-SNAPSHOT ........... SUCCESS [  2.176 s]
[INFO] Hive Shims Aggregator 4.0.0-SNAPSHOT ............... SUCCESS [  0.125 s]
[INFO] Hive Kryo Registrator 4.0.0-SNAPSHOT ............... SUCCESS [  3.806 s]
[INFO] Hive Kudu Handler 4.0.0-SNAPSHOT ................... SUCCESS [  7.185 s]
[INFO] Hive TestUtils 4.0.0-SNAPSHOT ...................... SUCCESS [  0.216 s]
[INFO] Hive Kafka Storage Handler 4.0.0-SNAPSHOT .......... SUCCESS [  4.713 s]
[INFO] Hive Packaging 4.0.0-SNAPSHOT ...................... SUCCESS [  4.690 s]
[INFO] Hive Metastore Tools 4.0.0-SNAPSHOT ................ SUCCESS [  0.024 s]
[INFO] Hive Metastore Tools common libraries 4.0.0-SNAPSHOT SUCCESS [ 11.620 s]
[INFO] Hive metastore benchmarks 4.0.0-SNAPSHOT ........... SUCCESS [ 13.379 s]
[INFO] Hive Upgrade Acid 4.0.0-SNAPSHOT ................... SUCCESS [  0.023 s]
[INFO] Hive Pre Upgrade Acid 4.0.0-SNAPSHOT ............... SUCCESS [  2.916 s]
[INFO] Hive Integration - Parent 4.0.0-SNAPSHOT ........... SUCCESS [  0.486 s]
[INFO] Hive Integration - Custom Serde 4.0.0-SNAPSHOT ..... SUCCESS [  2.189 s]
[INFO] Hive Integration - Custom udfs 4.0.0-SNAPSHOT ...... SUCCESS [  0.231 s]
[INFO] Hive Integration - Custom UDFs - udf-classloader-util 4.0.0-SNAPSHOT SUCCESS [  0.327
s]
[INFO] Hive Integration - Custom UDFs - udf-classloader-udf1 4.0.0-SNAPSHOT SUCCESS [  1.583
s]
[INFO] Hive Integration - Custom UDFs - udf-classloader-udf2 4.0.0-SNAPSHOT SUCCESS [  1.622
s]
[INFO] Hive Integration - Custom UDFs - udf-vectorized-badexample 4.0.0-SNAPSHOT SUCCESS [
 1.443 s]
[INFO] Hive Integration - HCatalog Unit Tests 4.0.0-SNAPSHOT SUCCESS [  5.993 s]
[INFO] Hive Integration - QFile Druid Tests 4.0.0-SNAPSHOT  SUCCESS [ 37.034 s]
[INFO] Hive Integration - Testing Utilities 4.0.0-SNAPSHOT  SUCCESS [ 12.331 s]
[INFO] Hive Integration - Unit Tests 4.0.0-SNAPSHOT ....... SUCCESS [ 12.586 s]
[INFO] Hive Integration - Blobstore Tests 4.0.0-SNAPSHOT .. SUCCESS [ 12.109 s]
[INFO] Hive Integration - Test Serde 4.0.0-SNAPSHOT ....... SUCCESS [  1.501 s]
[INFO] Hive Integration - QFile Tests 4.0.0-SNAPSHOT ...... SUCCESS [  6.351 s]
[INFO] Hive Integration - QFile Accumulo Tests 4.0.0-SNAPSHOT SUCCESS [  8.349 s]
[INFO] JMH benchmark: Hive 4.0.0-SNAPSHOT ................. SUCCESS [  4.702 s]
[INFO] Hive Integration - Unit Tests - Hadoop 2 4.0.0-SNAPSHOT SUCCESS [  5.005 s]
[INFO] Hive Integration - Unit Tests with miniKdc 4.0.0-SNAPSHOT SUCCESS [  9.543 s]
[INFO] Hive Integration - QFile Kudu Tests 4.0.0-SNAPSHOT . SUCCESS [  2.144 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  08:01 min
[INFO] Finished at: 2020-08-27T17:58:51+02:00
[INFO] ------------------------------------------------------------------------

{code}

> Optimize build times of druid related modules
> ---------------------------------------------
>
>                 Key: HIVE-24085
>                 URL: https://issues.apache.org/jira/browse/HIVE-24085
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: László Bodor
>            Priority: Major
>
> IMO druid modules relatively take long compared to others, I mean these below:
> {code}
> [INFO] Hive Druid Handler 4.0.0-SNAPSHOT .................. SUCCESS [ 19.663 s]
> [INFO] Hive Integration - QFile Druid Tests 4.0.0-SNAPSHOT  SUCCESS [ 37.034 s]
> {code}
> I accept a longer time only from core fundamental modules (e.g. hive-exec, metastore).
> {code}
>  mvn clean install -DskipTests -Pitests
> {code}
> {code}
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Hive Storage API 2.7.0-SNAPSHOT .................... SUCCESS [  7.520 s]
> [INFO] Hive 4.0.0-SNAPSHOT ................................ SUCCESS [  2.230 s]
> [INFO] Hive Classifications 4.0.0-SNAPSHOT ................ SUCCESS [  0.616 s]
> [INFO] Hive Shims Common 4.0.0-SNAPSHOT ................... SUCCESS [  3.686 s]
> [INFO] Hive Shims 0.23 4.0.0-SNAPSHOT ..................... SUCCESS [  6.012 s]
> [INFO] Hive Shims Scheduler 4.0.0-SNAPSHOT ................ SUCCESS [  2.360 s]
> [INFO] Hive Shims 4.0.0-SNAPSHOT .......................... SUCCESS [  1.806 s]
> [INFO] Hive Standalone Metastore 4.0.0-SNAPSHOT ........... SUCCESS [  1.489 s]
> [INFO] Hive Standalone Metastore Common Code 4.0.0-SNAPSHOT SUCCESS [ 26.412 s]
> [INFO] Hive Common 4.0.0-SNAPSHOT ......................... SUCCESS [  7.406 s]
> [INFO] Hive Service RPC 4.0.0-SNAPSHOT .................... SUCCESS [  3.394 s]
> [INFO] Hive Serde 4.0.0-SNAPSHOT .......................... SUCCESS [  6.063 s]
> [INFO] Hive Metastore 4.0.0-SNAPSHOT ...................... SUCCESS [  3.430 s]
> [INFO] Hive Vector-Code-Gen Utilities 4.0.0-SNAPSHOT ...... SUCCESS [  0.574 s]
> [INFO] Hive Parser 4.0.0-SNAPSHOT ......................... SUCCESS [ 12.159 s]
> [INFO] Hive UDF 4.0.0-SNAPSHOT ............................ SUCCESS [  1.250 s]
> [INFO] Hive Llap Common 4.0.0-SNAPSHOT .................... SUCCESS [  3.412 s]
> [INFO] Hive Llap Client 4.0.0-SNAPSHOT .................... SUCCESS [  1.882 s]
> [INFO] Hive Llap Tez 4.0.0-SNAPSHOT ....................... SUCCESS [  2.888 s]
> [INFO] Hive Spark Remote Client 4.0.0-SNAPSHOT ............ SUCCESS [  3.946 s]
> [INFO] Hive Metastore Server 4.0.0-SNAPSHOT ............... SUCCESS [ 20.209 s]
> [INFO] Hive Query Language 4.0.0-SNAPSHOT ................. SUCCESS [01:06 min]
> [INFO] Hive Llap Server 4.0.0-SNAPSHOT .................... SUCCESS [  9.777 s]
> [INFO] Hive Service 4.0.0-SNAPSHOT ........................ SUCCESS [  8.090 s]
> [INFO] Hive Accumulo Handler 4.0.0-SNAPSHOT ............... SUCCESS [  7.768 s]
> [INFO] Hive JDBC 4.0.0-SNAPSHOT ........................... SUCCESS [ 24.253 s]
> [INFO] Hive Beeline 4.0.0-SNAPSHOT ........................ SUCCESS [  4.741 s]
> [INFO] Hive CLI 4.0.0-SNAPSHOT ............................ SUCCESS [  2.852 s]
> [INFO] Hive Contrib 4.0.0-SNAPSHOT ........................ SUCCESS [  1.648 s]
> [INFO] Hive Druid Handler 4.0.0-SNAPSHOT .................. SUCCESS [ 19.663 s]
> [INFO] Hive HBase Handler 4.0.0-SNAPSHOT .................. SUCCESS [  5.218 s]
> [INFO] Hive JDBC Handler 4.0.0-SNAPSHOT ................... SUCCESS [  2.004 s]
> [INFO] Hive HCatalog 4.0.0-SNAPSHOT ....................... SUCCESS [  0.764 s]
> [INFO] Hive HCatalog Core 4.0.0-SNAPSHOT .................. SUCCESS [  4.746 s]
> [INFO] Hive HCatalog Pig Adapter 4.0.0-SNAPSHOT ........... SUCCESS [  2.959 s]
> [INFO] Hive HCatalog Server Extensions 4.0.0-SNAPSHOT ..... SUCCESS [  2.381 s]
> [INFO] Hive HCatalog Webhcat Java Client 4.0.0-SNAPSHOT ... SUCCESS [  2.970 s]
> [INFO] Hive HCatalog Webhcat 4.0.0-SNAPSHOT ............... SUCCESS [ 12.019 s]
> [INFO] Hive HPL/SQL 4.0.0-SNAPSHOT ........................ SUCCESS [  4.274 s]
> [INFO] Hive Streaming 4.0.0-SNAPSHOT ...................... SUCCESS [  2.357 s]
> [INFO] Hive Llap External Client 4.0.0-SNAPSHOT ........... SUCCESS [  2.176 s]
> [INFO] Hive Shims Aggregator 4.0.0-SNAPSHOT ............... SUCCESS [  0.125 s]
> [INFO] Hive Kryo Registrator 4.0.0-SNAPSHOT ............... SUCCESS [  3.806 s]
> [INFO] Hive Kudu Handler 4.0.0-SNAPSHOT ................... SUCCESS [  7.185 s]
> [INFO] Hive TestUtils 4.0.0-SNAPSHOT ...................... SUCCESS [  0.216 s]
> [INFO] Hive Kafka Storage Handler 4.0.0-SNAPSHOT .......... SUCCESS [  4.713 s]
> [INFO] Hive Packaging 4.0.0-SNAPSHOT ...................... SUCCESS [  4.690 s]
> [INFO] Hive Metastore Tools 4.0.0-SNAPSHOT ................ SUCCESS [  0.024 s]
> [INFO] Hive Metastore Tools common libraries 4.0.0-SNAPSHOT SUCCESS [ 11.620 s]
> [INFO] Hive metastore benchmarks 4.0.0-SNAPSHOT ........... SUCCESS [ 13.379 s]
> [INFO] Hive Upgrade Acid 4.0.0-SNAPSHOT ................... SUCCESS [  0.023 s]
> [INFO] Hive Pre Upgrade Acid 4.0.0-SNAPSHOT ............... SUCCESS [  2.916 s]
> [INFO] Hive Integration - Parent 4.0.0-SNAPSHOT ........... SUCCESS [  0.486 s]
> [INFO] Hive Integration - Custom Serde 4.0.0-SNAPSHOT ..... SUCCESS [  2.189 s]
> [INFO] Hive Integration - Custom udfs 4.0.0-SNAPSHOT ...... SUCCESS [  0.231 s]
> [INFO] Hive Integration - Custom UDFs - udf-classloader-util 4.0.0-SNAPSHOT SUCCESS [
 0.327 s]
> [INFO] Hive Integration - Custom UDFs - udf-classloader-udf1 4.0.0-SNAPSHOT SUCCESS [
 1.583 s]
> [INFO] Hive Integration - Custom UDFs - udf-classloader-udf2 4.0.0-SNAPSHOT SUCCESS [
 1.622 s]
> [INFO] Hive Integration - Custom UDFs - udf-vectorized-badexample 4.0.0-SNAPSHOT SUCCESS
[  1.443 s]
> [INFO] Hive Integration - HCatalog Unit Tests 4.0.0-SNAPSHOT SUCCESS [  5.993 s]
> [INFO] Hive Integration - QFile Druid Tests 4.0.0-SNAPSHOT  SUCCESS [ 37.034 s]
> [INFO] Hive Integration - Testing Utilities 4.0.0-SNAPSHOT  SUCCESS [ 12.331 s]
> [INFO] Hive Integration - Unit Tests 4.0.0-SNAPSHOT ....... SUCCESS [ 12.586 s]
> [INFO] Hive Integration - Blobstore Tests 4.0.0-SNAPSHOT .. SUCCESS [ 12.109 s]
> [INFO] Hive Integration - Test Serde 4.0.0-SNAPSHOT ....... SUCCESS [  1.501 s]
> [INFO] Hive Integration - QFile Tests 4.0.0-SNAPSHOT ...... SUCCESS [  6.351 s]
> [INFO] Hive Integration - QFile Accumulo Tests 4.0.0-SNAPSHOT SUCCESS [  8.349 s]
> [INFO] JMH benchmark: Hive 4.0.0-SNAPSHOT ................. SUCCESS [  4.702 s]
> [INFO] Hive Integration - Unit Tests - Hadoop 2 4.0.0-SNAPSHOT SUCCESS [  5.005 s]
> [INFO] Hive Integration - Unit Tests with miniKdc 4.0.0-SNAPSHOT SUCCESS [  9.543 s]
> [INFO] Hive Integration - QFile Kudu Tests 4.0.0-SNAPSHOT . SUCCESS [  2.144 s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time:  08:01 min
> [INFO] Finished at: 2020-08-27T17:58:51+02:00
> [INFO] ------------------------------------------------------------------------
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message