uima-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tommaso Teofili <tommaso.teof...@gmail.com>
Subject Re: UIMA : java.lang.OutOfMemoryError: Java heap space.....
Date Sun, 08 Mar 2009 14:58:04 GMT
2009/3/8 Marshall Schor <msa@schor.com>

> Balkrishnan.V wrote:
> > Marshall Schor <msa@...> writes:
> >
> >
> >> Did you try increasing your java heap size?  What size are you running
> with?
> >>
> >> -Marshall
> >>
> >>
> >
> > Hi Marshall,
> >
> > ThankYou for your time.
> >
> > Yes, I tried resetting the jvm-heap-size to 1327MB and now I am able to
> run the
> > CVD successfully with 101MB text-log-file.
> >
> > Can you please let me know :
> > (i) Is this maximum heap-size setting (1327MB) enough to handle a
> text-file of
> > size 2GB(say) ?
> It depends on what you're running in UIMA.  But, since 1327MB = about
> 1.3 GB, and that's less than the size of the text file you're hoping to
> process (2GB), maybe it it isn't big enough.  But, it really depends on
> the components you're running within UIMA.  For instance, you might be
> running components which you've written, which read a small part of the
> 2GB text file, as a "chunk", and then process that through the pipeline
> of UIMA components, and then the next small chunk could be processed,
> etc.  It really just depends on the particular processing you're doing.
> > If not, then any idea the approximate maximum text-file size
> > that can be handled with this heap-size setting ?
> >
> No, again it depends not on UIMA, but more on the components UIMA is
> running as annotators.
> > (ii) Why I am unable to increase the JVM Heap-Size more than 1327MB ?
> >
> I don't know.  It may be a limitation of your particular machine.  We
> have seen users who have 64-bit linux machines, running with heap sizes
> of 16 GB, for instance.


This one could be the common issue with JVM and 32 bit architectures: java
heap space is required to be contiguous but, due to 32 bit processor, only
4Gb of maximum contiguous memory can be addressed so, depending on the OS
and the physical memory you have, the amount of addressable user-space
memory can be much less of the total available physical memory.
Claiming this problem to Sun's JVM developers I simply obtained this answer:
"Buy a 64-bit machine"; this implies they don't want to make possible to
address non contiguous space for the JVM.
So, as Marshall said, I also think that splitting your big problem in some
lighter problems could be the solution.
Regards,
Tommaso



>
> > (iii) What should I do to handle the text-files that cannot be handled
> within
> > the bounds of the heap-size setting of 1327MB ?
> >
> A good approach is to break these things up into more managable
> work-units, and process each one separately.
>
> -Marshall
> > Kind Regards,
> > Balkrishnan.V
> >
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message