giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-1181) Changes to ExtendedByteArrayDataOutput
Date Thu, 29 Mar 2018 16:28:00 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419298#comment-16419298
] 

ASF GitHub Bot commented on GIRAPH-1181:
----------------------------------------

Github user yukselakinci commented on a diff in the pull request:

    https://github.com/apache/giraph/pull/65#discussion_r178111159
  
    --- Diff: giraph-core/src/main/java/org/apache/giraph/comm/aggregators/CountingOutputStream.java
---
    @@ -36,7 +36,7 @@
        * Default constructor
        */
       public CountingOutputStream() {
    -    dataOutput = new ExtendedByteArrayDataOutput();
    +    dataOutput = new UnsafeByteArrayOutputStream();
    --- End diff --
    
    These changes allow us to write to the underlying buffer without having to first write
to Chunked stream. However, what we write to stream is different depending on if the temporary
Chunked streams is used, since the chunk size is also written. So we need to be consistent
between write and read for kryo serialization. If the temporary chunk is not used when writing,
casting the byte stream to ordinary ByteArrayInputStream when reading,  would cause error
as we assume it used chunked stream. Since we know all our streams derives from UnsafeByteArray[Input/Outut]Stream,
we can safely cast the byte array to unsafeByteArray stream.


> Changes to ExtendedByteArrayDataOutput
> --------------------------------------
>
>                 Key: GIRAPH-1181
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-1181
>             Project: Giraph
>          Issue Type: Improvement
>            Reporter: Yuksel Akinci
>            Priority: Major
>
> Adding a new stream type which extends from com.esotericsoftware.kryo.io.Input/Output,
so that it can be used with kryo serializer. It improves the performance of kryo by faster
read/writes (unsafe IO), and also eliminates the need for interim buffers to convert from DataInput
and DataOutput.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message