lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shawn Heisey (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (LUCENE-5928) WildcardQuery may has memory leak
Date Fri, 19 Sep 2014 03:18:33 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14139919#comment-14139919
] 

Shawn Heisey edited comment on LUCENE-5928 at 9/19/14 3:18 AM:
---------------------------------------------------------------

I have seen this exact memory reporting issue on my Solr installs.  Java appears to misreport
memory usage.  See how your SHR value is 37GB?  If you subtract that from your RES value,
I think that's a lot closer to how much memory it's actually using.

For what follows, reference the screenshot that I have attached.

!idxb1-top-sorted-mem.png!

Solr on this machine has a max heap of 6144M.  The machine has 64GB of RAM.  Let's add up
some numbers, each of which I would ordinarily consider to be absolute truth:

16GB: RES size of the Solr process.
9.8GB: The amount of memory listed as free.
44GB: The amount of memory used by the OS disk cache.
1.1GB: next largest java process.
0.3GB: next largest java process.

These numbers add up to more than 70GB ... but the machine only has 64GB total.  But if you
subtract the 11GB value in the SHR column, then it all fits, and the resulting number is also
less than the max heap of 6144M.



was (Author: elyograg):
I have seen this exact memory reporting issue on my Solr installs.  Java appears to misreport
memory usage.  See how your SHR value is 37GB?  If you subtract that from your RES value,
I think that's a lot closer to how much memory it's actually using.

For what follows, reference the screenshot that I have attached.

!idxb1-top-sorted-mem.png!

Solr on this machine has a max heap of 6144M.  The machine has 64GB of RAM.  Let's add up
some numbers, each of which I would ordinarily consider to be absolute truth:

16GB: RES size of the Solr process.
9.8GB: The amount of memory listed as free.
44GB: The amount of memory used by the OS disk cache.
1.1GB: next largest java process.
0.3GB: next largest java process.

These numbers add up to more than 70GB ... but the machine only has 64GB total.



> WildcardQuery may has memory leak
> ---------------------------------
>
>                 Key: LUCENE-5928
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5928
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/search
>    Affects Versions: 4.9
>         Environment: SSD 1.5T, RAM 256G, records 15*10000*10000
>            Reporter: Littlestar
>            Assignee: Uwe Schindler
>         Attachments: idxb1-top-sorted-mem.png, top_java.jpg
>
>
> data 800G, records 15*10000*10000.
> one search thread.
> content:???
> content:*
> content:*1
> content:*2
> content:*3
> jvm heap=96G, but the jvm memusage over 130g?
> run more wildcard, use memory more....
> Does luence search/index use a lot of DirectMemory or Native Memory?
> I use -XX:MaxDirectMemorySize=4g, it does nothing better.
> Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message