directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefan Seelmann (JIRA)" <>
Subject [jira] Commented: (DIRSTUDIO-327) Add support for Paged Results Control
Date Wed, 03 Sep 2008 11:53:44 GMT


Stefan Seelmann commented on DIRSTUDIO-327:

I think I was able to reproduce what you done: You first searched 13K entries and they were
displayed in the search result editor, then you exported them, right?

Perfoming a search with 10K results and displaying them in the search result editor costs
about 50MB memory. The search results are cached in memory as long as your connection is opened.
5kB for one entry sounds much, it consists of our internal object model (entry, DN, RDN, attributes,
values, parent-child relationship) and the UI objects to display our object model (Table,
Rows, Column, Fonts)

Exporting 10K entries to Excel costs even about 50MB memory. We use the Apache POI library
for that and we must create the excel file in memory.

Exporting to LDIF or CSV is cheaper because each entry received from the server is immediately
streamed to the file. (With LDIF the CPU usage is too high, need to check that...)

So 40MB (Studio/Eclipse footprint) + 50MB (10000 search results) + 50MB (Excel export) is
more the 128MB default heap size.

So what I could suggest (if appropriate)
- you already increased heap memory
- only perform small searches within Studio, use the count limit and/or paged search
- If you need to export large data
  - use CSV (you already do) or LDIF 
  - if you need excel first close all connections (to flush caches), open the right connection,
run the export without performing a search

Hm, perhaps we should start a new process for each search and export, like g**gle does with
chr*me ;-)

> Add support for Paged Results Control
> -------------------------------------
>                 Key: DIRSTUDIO-327
>                 URL:
>             Project: Directory Studio
>          Issue Type: Improvement
>    Affects Versions: 1.2.0
>         Environment: Win2k workstation, java 1.6.0_05
>            Reporter: Jim Birch
>            Assignee: Stefan Seelmann
>             Fix For: 1.3.0
> Windows servers have a default server side limit of 1000 returned objects.  AFAIK the
normal way of handling this is to detect that a search returns an incomplete result set and
make further requests to span the full result set.
> There appears to be no such capability in dirstudio which makes searching 15K users extremely

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message