hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6381) WritableUtils::*VLong utilities should be available for byte arrays
Date Wed, 18 Nov 2009 21:24:39 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12779676#action_12779676

Chris Douglas commented on HADOOP-6381:

bq. Doesn't it avoid the need to construct a ByteArrayInputStream?

Yes, but if one has a DataInputBuffer for the purpose, then I wouldn't expect it to be too
expensive. Given a key type with two vint params, to sort by the second:
final DataInputBuffer di = new DataInputBuffer(0);

int compare(byte[] b1, int s1, int l1, byte[] b2, int s2, int l2) {
  di.reset(b1, s1, l1);
  final int v1 = WritableUtils.readVInt(di);
  final int x1 = WritableUtils.decodeVIntSize(b1[s1]);
  // second param starts at b[s1 + v1 + x1]

Which isn't too expensive or inelegant. Thread safety and a memory leak from the DIB reference
can both be handled, again, without imposing too high a penalty, even for an inner loop. It
just struck me as a lot of ceremony.

> WritableUtils::*VLong utilities should be available for byte arrays
> -------------------------------------------------------------------
>                 Key: HADOOP-6381
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6381
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: util
>            Reporter: Chris Douglas
>            Priority: Minor
>         Attachments: C6381-0.patch, C6381-1.patch, C6381-2.patch
> Particularly when working with raw bytes in Writables, it is often useful to have versions
of the vint utility functions for byte arrays.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message