db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Ondruška <peter.ondru...@kaibo.eu>
Subject Re: Page cache sizing
Date Sun, 09 Jul 2017 17:56:20 GMT
Thanks Bryan,

I have done different kind of test so far and this is my observation:

1. set network server JVM heap size to 512 M
2. set parameter derby.storage.pageCacheSize=16384
3. booted a database with mixed block sizes (default 4 K, and large 32 K)
4. after some time I get OOM when conneting network client, this OOM is
reported back from network server, see * and ** (tried to backup as well)
5. only changed parameter derby.storage.pageCacheSize=4096
6. no more OOM

Conclusion seems to be that pageCacheSize may be based really on just the
number of cached blocks (as documented) and if they are mixed you better
size based on 32 K size. If this is confirmed by the developers I would
suggest ammending documentation with such note because some objects may
automatically use 32 K page (LOBs) and can cause OOM suddenly if
pageCacheSize was set for 4 K block size. But ideally a separate
pageCacheSize setting would be great for large/non-standard block size(s).

*
Sun Jul 09 00:24:20 CEST 2017 Thread[DRDAConnThread_3,5,main] (DATABASE =
store), (DRDAID = {233}), Connection refused : java.lang.OutOfMemoryError
Sun Jul 09 01:02:48 CEST 2017 Thread[DRDAConnThread_7,5,main] (DATABASE =
store), (DRDAID = {266}), Connection refused : java.lang.OutOfMemoryError
Sun Jul 09 01:24:11 CEST 2017 Thread[DRDAConnThread_6,5,main] (DATABASE =
store), (DRDAID = {283}), Connection refused : java.lang.OutOfMemoryError
Sun Jul 09 02:24:12 CEST 2017 Thread[DRDAConnThread_7,5,main] (DATABASE =
store), (DRDAID = {332}), Connection refused : java.lang.OutOfMemoryError
Sun Jul 09 03:24:11 CEST 2017 Thread[DRDAConnThread_4,5,main] (DATABASE =
store), (DRDAID = {381}), Connection refused : java.lang.OutOfMemoryError
Sun Jul 09 04:24:10 CEST 2017 Thread[DRDAConnThread_10,5,main] (DATABASE =
store), (DRDAID = {430}), Connection refused : java.lang.OutOfMemoryError
Sun Jul 09 04:30:30 CEST 2017 Thread[DRDAConnThread_9,5,main] (DATABASE =
store), (DRDAID = {439}), Connection refused : java.lang.OutOfMemoryError
Sun Jul 09 04:30:31 CEST 2017 Thread[DRDAConnThread_9,5,main] (DATABASE =
store), (DRDAID = {440}), Connection refused : java.lang.OutOfMemoryError
Sun Jul 09 05:24:11 CEST 2017 Thread[DRDAConnThread_3,5,main] (DATABASE =
store), (DRDAID = {481}), Connection refused : java.lang.OutOfMemoryError
Sun Jul 09 06:24:12 CEST 2017 Thread[DRDAConnThread_2,5,main] (DATABASE =
store), (DRDAID = {530}), Connection refused : java.lang.OutOfMemoryError
Sun Jul 09 07:24:12 CEST 2017 Thread[DRDAConnThread_10,5,main] (DATABASE =
store), (DRDAID = {579}), Connection refused : java.lang.OutOfMemoryError
Sun Jul 09 07:24:45 CEST 2017 Thread[DRDAConnThread_3,5,main] (DATABASE =
ejbtimer), (DRDAID = {582}), Connection refused : java.lang.OutOfMemoryError

**
 ERROR XJ001: Java exception: 'java.lang.OutOfMemoryError: Java heap space'.
        at
org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
        at
org.apache.derby.iapi.error.StandardException.plainWrapException(Unknown
Source)
        at
org.apache.derby.impl.store.raw.RawStore.backupAndEnableLogArchiveMode(Unknown
Source)
        at
org.apache.derby.impl.store.access.RAMAccessManager.backupAndEnableLogArchiveMode(Unknown
Source)
        at
org.apache.derby.impl.db.BasicDatabase.backupAndEnableLogArchiveMode(Unknown
Source)
        at
org.apache.derby.catalog.SystemProcedures.SYSCS_BACKUP_DATABASE_AND_ENABLE_LOG_ARCHIVE_MODE(Unknown
Source)
        at
org.apache.derby.exe.ac2a2882b5x015dx2354xca08xffff89e47f210.g0(Unknown
Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
        at java.lang.reflect.Method.invoke(Method.java:508)
        at
org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source)
        at
org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown
Source)
        at
org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown
Source)
        at
org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
        at
org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
        at
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown
Source)
        at
org.apache.derby.impl.jdbc.EmbedCallableStatement.executeStatement(Unknown
Source)
        at
org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
        at org.apache.derby.impl.drda.DRDAStatement.execute(Unknown Source)
        at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(Unknown
Source)
        at
org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
        at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
Caused by: java.lang.OutOfMemoryError: Java heap space
        at
org.apache.derby.impl.store.raw.data.CachedPage.setPageArray(Unknown Source)
        at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown
Source)
        at
org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source)
        at
org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown Source)
        at
org.apache.derby.impl.store.raw.data.FileContainer.getLatchedPage(Unknown
Source)
        at
org.apache.derby.impl.store.raw.data.RAFContainer.backupContainer(Unknown
Source)
        at
org.apache.derby.impl.store.raw.data.BaseContainerHandle.backupContainer(Unknown
Source)
        at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.backupDataFiles(Unknown
Source)
        at org.apache.derby.impl.store.raw.RawStore.backup(Unknown Source)
        at org.apache.derby.impl.store.raw.RawStore.backup(Unknown Source)

Regards,
p.




On 7 July 2017 at 04:59, Bryan Pendleton <bpendleton.derby@gmail.com> wrote:

>
>
> how does derby.storage.pageCacheSize parameter (
>> https://db.apache.org/derby/docs/10.13/ref/rrefproper81359.html) work
>> with database that has multiple page sizes--tables with default 4096 bytes
>> and tables with long/blob of 32768 byte pages?
>>
>>
> Hi Peter,
>
> I'm not 100% sure how this works; I think you should run some experiments.
>
> Here's what I *think* the behavior is:
> 1) The page cache is sized in 4K pages, so if you set pageCacheSize=1024,
> you get 4 meg of page cache memory
> 2) Tables with 4K pages simply use the cache as you expect.
> 3) Tables with 32K pages chew up 8 cache "pages" at a time, each 32K chunk
> of page cache holding 1 32K page from that large-page table.
>
> Some stuff I found while searching around, which might give you some ideas
> for experiments you could run:
>
> http://apache-database.10148.n7.nabble.com/out-of-memory-
> when-writing-blobs-td100948.html
>
> and
>
> https://issues.apache.org/jira/browse/DERBY-4537
>
> Sorry I'm not of much more help here.
>
> bryan
>
>


-- 
Peter Ondruška

-- 
kaibo, s.r.o., ID 28435036, registered with the commercial register 
administered by the Municipal Court in Prague, section C, insert 141269.
Registered office and postal address: kaibo, s.r.o., Kališnická 379/10, 
Prague 3, 130 00, Czech Republic.
https://www.kaibo.eu

Mime
View raw message