directory-api mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel Lécharny <>
Subject Re: Binary values and humanRedable flag
Date Mon, 10 Aug 2015 13:10:39 GMT
Le 10/08/15 13:33, Radovan Semancik a écrit :
> On 08/10/2015 12:42 PM, Emmanuel Lécharny wrote:
>> There is no flag that says an Attribute is H-R or not. The
>> information is provided in RFC 22524.3.2
>> <>
> Hmm, I was code for parsing of "X-NOT-HUMAN-READABLE" so I thought
> that it might be caused by this. Thanks for clarification. Anyway, the
> strange thing is that the syntax appears
> to be human readable.

WHich it is not :

version: 1
dn: m-oid=,ou=syntaxes,cn=system,ou=schema
objectclass: top
objectclass: metaTop
objectclass: metaSyntax
m-description: JPEG
m-obsolete: FALSE
x-not-human-readable: TRUE
entrycsn: 20100111202214.878000Z#000000#000#000000
creatorsname: uid=admin,ou=system
createtimestamp: 20100111145217Z

Actually, when we load the schema, we read the associated LDIF entries
and convert them to schema element instance. If the LDIF entry contains
the x-not-human-readable flag, then the Syntax will be set to not H-R.

>>> But then jpegPhoto is obviously binary. When I try to modify it with
>>> BinaryValue it ends up with:
>>> ERROR (o.a.d.api.ldap.model.entry.DefaultAttribute): ERR_04451 The
>>> value must be a String, as its AttributeType is H/R
>>> I'm doing the equivalent of attribute.add(new BinaryValue(bytes));
>> Hmmm... Do you have the full code as a test I can run on my machine ?
> Try this:
>             LdapNetworkConnection conn = new
> LdapNetworkConnection("localhost", 10389);
>             DefaultSchemaLoader schemaLoader;
>                 schemaLoader = new DefaultSchemaLoader(conn, true);
>             DefaultSchemaManager schemaManager = new
> DefaultSchemaManager(schemaLoader);
>             schemaManager.setRelaxed();
>             schemaManager.loadAllEnabledRelaxed();
>             AttributeType jpegPhotoAttributeType =
> schemaManager.getAttributeType("jpegPhoto");
>             System.out.println("jpegPhoto AttributeType =
> "+jpegPhotoAttributeType);
>             LdapSyntax syntax = jpegPhotoAttributeType.getSyntax();
>             System.out.println("jpegPhoto syntax = "+syntax);
>             System.out.println("jpegPhoto syntax H/R =
> "+syntax.isHumanReadable());

Here is what I get when connected to an ApacheDS server :

jpegPhoto AttributeType = attributetype ( 0.9.2342.19200300.100.1.60
NAME 'jpegPhoto'
    DESC 'RFC2798: a JPEG image'
    USAGE userApplications )
jpegPhoto syntax = ldapsyntax (
jpegPhoto syntax H/R = false

Seems correct to me...

> But there is a very strange thing going on. Trying it with OpenDJ
> (2.6) I got this:
> jpegPhoto AttributeType = attributetype ( 0.9.2342.19200300.100.1.60
> NAME 'jpegPhoto'
>     DESC 'a JPEG image'
>     SYNTAX
>     USAGE userApplications )
> jpegPhoto syntax = ldapsyntax (
>     DESC 'JPEG'
>     X-NOT-HUMAN-READABLE 'false' )
> jpegPhoto syntax H/R = true

X-NOT-HUMAN-READABLE 'false', which means it's hulan readable. But I
guess OpenDJ does *not* set the X-NOT-HUMAN-READABLE flag, while
openLDAP does.

I expect the server or the client to *know* magically that this
attribute is H/R when connected to OpenDJ, right ? (irony)

> But running it with OpenLDAP (2.4.31) I got this:
> jpegPhoto AttributeType = attributetype ( 0.9.2342.19200300.100.1.60
> NAME 'jpegPhoto'
>     DESC 'RFC2798: a JPEG image'
>     SYNTAX
>     USAGE userApplications )
> jpegPhoto syntax = ldapsyntax (
>     DESC 'JPEG'
>     X-NOT-HUMAN-READABLE 'true' )
> jpegPhoto syntax H/R = false
> The attribute schema definition is almost identical:
> OpenDJ
> attributeTypes: ( 0.9.2342.19200300.100.1.60 NAME 'jpegPhoto' DESC 'a
> JPEG ima
>  ge' SYNTAX X-ORIGIN 'RFC 2798' )
> OpenLDAP
> attributeTypes: ( 0.9.2342.19200300.100.1.60 NAME 'jpegPhoto' DESC
> 'RFC2798: a
>   JPEG image' SYNTAX )
> Could the X-ORIGIN make a difference here?

No. OpenLDAP defines the H-R (or non-H)R) attributes in teh code :

    {"( DESC 'JPEG' " X_NOT_H_R ")",
        SLAP_SYNTAX_BLOB, NULL, blobValidate, NULL},

This get added to the description when you ask teh server about the

>>> And one related remark. The add( Value<?>... vals ) method in
>>> DefaultAttribute is not very convenient to use and still keep a good
>>> error reporting. E.g. in the above case it will not throw any error,
>>> just returns zero. This is enough to detect that there was an error
>>> (not very convenient though). But the reason why the attribute cannot
>>> be added is lost. I cannot report back to the higher layers anything
>>> better than "failed to add foo to bar". But the user will not know
>>> what was the reason.
>> Yes, that's true. the rational is that we do a best effort to inject
>> values correctly, converting them on the fly.
>> Note that this H-R flag itself is stupid. It was added 12 years ago as a
>> way to follow teh RFC, but as a matter of fact, the Syntax itself
>> already drives the type of data we can store in an Attribute. I made it
>> even more complex by trying to use Generics. Now, we have those
>> StringValue and BinaryValue all over the code.
>> Ideally, we should not have to care about what we store, and always
>> consider the stored values as byte[]. OTOH, it's not convenient when we
>> want to manipulate values as String, as converting them over and over
>> from byte[] to Strings is costly (epecially in the server). But I do
>> think we went way to far here. This conversion should be done internally
>> once, and that's it. It would save us a hell lot of time, and would make
>> the APi more comfortable to use.
> I tend to agree. Always storing the value as binary seems to be good
> idea.

Depends. from the performance POV, this is killing the server. Most of
the AT are H/R, and require some checks (comparison, normalization, etc)
during the processing of every request. Having only the binary value is
forcing the server to do the conversion back and forth multiple times.
We faced this issue and when we switched to StringValue and BinaryValue,
the performance boost was huge (100%).

Ideally, we should have 2 methods :
- getBinaryValue()
- getStringValue()

because we always know which type we are dealing with. But that's the
point : in the server, for operatiuons involving many attributes, that
would require a check on the Syntax everytime we want to manipulate a
value, which is a bit of a PITA, especially when we don't care about
this type. Having a Value<?> wrapper helps a lot here...

> I'm working on a similar problem with Novell/NetIQ eDirectory. eDir
> has a binary GUID. But the API seems to try to interpret it as String.
> As it contains some invalid UTF codes it completely breaks the value
> and there is no way how to fix this on the API client side. Storing
> the value as binary will solve this kind of issues. This specific eDir
> GUID problem is probably my fault as the Studio seems to work with the
> GUID without any problems. I'll report on that later when I
> investigate it. Anyway, the API is currently not very intuitive with
> the BinaryValue, StringValue and H/R flags. Simplification will be a
> great help for new API users.

I'm really willing to find a better solution, I have worked a full
quarter on this issue (bin/string values) and I haven't be able to come
with something that hide the inconsitency and complexity of LDAP in this
area, sadly... May be it's time for a rehearsal...

View raw message