lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dr <bfore...@126.com>
Subject Re:Re:Re: IO probleam in forceMerge(1)
Date Fri, 21 Aug 2015 08:59:12 GMT


I used to say "when i did some copy operation on the machine the speed may become normal.",
 it seems not correct. But when forceMerge hangs on IO, if i use the command "echo 3 >
/proc/sys/vm/drop_caches", the io become normal again.
At 2015-08-21 11:59:30, "dr" <bforevdr@126.com> wrote:
>
>Also i noticed that in the begin the speed was normal, and  it slowed down  later. The
total space of the index is 15GB, and *.fdt takes 12GB. I allocated 8GB memory for forceMerge,
is this enough for that? I readed the code and doc, it seems that *.fdt is readed and  stored
in block of 16KB, i guess it won't need too much memory to merge them. 
>
>
>At 2015-08-20 22:00:27, "Michael McCandless" <lucene@mikemccandless.com> wrote:
>>I would first try upgrading you JVM: 1.7.0_05 is ancient.
>>
>>Mike McCandless
>>
>>http://blog.mikemccandless.com
>>
>>
>>On Thu, Aug 20, 2015 at 7:49 AM, dr <bforevdr@126.com> wrote:
>>> Hi ,all
>>>     Currenty i met a probleam with forceMerge(1). During forceMerging some of
my machine spent too much time(10-20 hours for a index size of 15GB). By using some command
like top, iostat, jstack. I found that the avg of cpu, and disk is too low, nearly zero. 
And the size of the the index files increased very very slowly. From jstack, i found that
the application is hanging on IO operations, like FileInputStream or DirectByteArray.  But
when i did some copy operation on the machine the speed may become normal.
>>>
>>>     Should this issue be related with the JVM, Operating System and hardware?
>>>     Did anybody met the same probleam, or have any idea about Java, Linux and
IO.
>>>     Some basic info about my system and application.
>>>
>>> |  Lucece version | 4.10.3 |
>>> | Jdk version |  jdk 1.7.0_05 |
>>> | OS |   redhat 4.4.4 |
>>> | cpu |      IntelX5650     |
>>> | Memory | 24GB |
>>>
>>>
>>> Best Reagrds
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>For additional commands, e-mail: java-user-help@lucene.apache.org
>>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message