hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "fanshilun (Jira)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-22410) CachedStore Prewarm Failure NPE
Date Fri, 01 Nov 2019 03:15:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-22410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964567#comment-16964567
] 

fanshilun edited comment on HIVE-22410 at 11/1/19 3:14 AM:
-----------------------------------------------------------

[~samuelan]
The problem does not appear in the Sql query, but appears in the HMS startup warm-up,I will
give some detailed instructions.
1.The problem occurs in the hot load function prewarm (RawStore rawStore) of the CachedStore.
After the DataBase is loaded, the table information loading starts.
2.sharedCache.populateTableInCache(table, tableColStats...) CachedStore 559 Lines Throw NPE.
3.Method populateTableInCache calculates the memory of the object, and throws NPE when performing
OBJECT_ARRAY calculation.
 !image-2019-11-01-11-14-42-114.png! 



was (Author: slfan1989):
[~samuelan]
The problem does not appear in the Sql query, but appears in the HMS startup warm-up,I will
give some detailed instructions.
1.The problem occurs in the hot load function prewarm (RawStore rawStore) of the CachedStore.
After the DataBase is loaded, the table information loading starts.
2.sharedCache.populateTableInCache(table, tableColStats...) CachedStore 559 Lines Throw NPE.
3.Method populateTableInCache calculates the memory of the object, and throws NPE when performing
OBJECT_ARRAY calculation.


> CachedStore Prewarm Failure NPE
> -------------------------------
>
>                 Key: HIVE-22410
>                 URL: https://issues.apache.org/jira/browse/HIVE-22410
>             Project: Hive
>          Issue Type: Bug
>          Components: Standalone Metastore
>    Affects Versions: 3.1.2, 2.3.7
>         Environment: JDK 1.7.0_40
>            Reporter: fanshilun
>            Assignee: fanshilun
>            Priority: Major
>              Labels: cachedstore
>             Fix For: 2.3.7, 3.1.3
>
>         Attachments: HIVE-22410.1.patch, image-2019-11-01-11-14-42-114.png
>
>
> java.lang.NullPointerException
> at org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimateArrayElements(IncrementalObjectSizeEstimator.java:501)
~[hive-metastore-2.3.5.jar:2.3.5]
> at org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimate(IncrementalObjectSizeEstimator.java:443)
~[hive-metastore-2.3.5.jar:2.3.5]
> at org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimate(IncrementalObjectSizeEstimator.java:433)
~[hive-metastore-2.3.5.jar:2.3.5]
> at org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimate(IncrementalObjectSizeEstimator.java:369)
~[hive-metastore-2.3.5.jar:2.3.5]
> at org.apache.hadoop.hive.metastore.cached.SharedCache.populateTableInCache(SharedCache.java:1028)
~[hive-metastore-2.3.5.jar:2.3.5]
> at org.apache.hadoop.hive.metastore.cached.CachedStore.prewarm(CachedStore.java:314)
~[hive-metastore-2.3.5.jar:2.3.5]
> at org.apache.hadoop.hive.metastore.cached.CachedStore$CacheUpdateMasterWork.run(CachedStore.java:492)
[hive-metastore-2.3.5.jar:2.3.5]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [?:1.7.0_40]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [?:1.7.0_40]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
[?:1.7.0_40]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[?:1.7.0_40]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_40]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_40]
> at java.lang.Thread.run(Thread.java:724) [?:1.7.0_40]
> *reason:*
> Methods estimateArrayElements throw NPE
> I found that the last class does not appear in the parent (HashMap<Class<?>,
Object Estimator> ), and the parent does not contain the information of the ConcurrentHashMap
Class.
> The solution is to initialize the class information before using it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message