thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randy Abernethy <randy.aberne...@gmail.com>
Subject Re: Error with binary data with node.js
Date Sat, 20 Sep 2014 23:03:43 GMT
I think the official answer to that is always "real soon now".

In the mean time you can get the current master with npm as follows:

$ npm install RandyAbernethy/thriftnode

This is a clone of the thrift master branch setup to work with npm.


On Fri, Sep 19, 2014 at 4:41 PM, Rodolfo Ochoa <rodolfo.ochoa@gmail.com> wrote:
> Oh, it is fixed on newest version...
>
> do you have a release date?
>
> thanks
>
> On Fri, Sep 19, 2014 at 6:24 PM, Rodolfo Ochoa <rodolfo.ochoa@gmail.com>
> wrote:
>
>> this error start happening here:
>> https://issues.apache.org/jira/browse/THRIFT-1679
>> which is a correct fix, but then you need to correctly implement binary.
>>
>> -Rod.O
>>
>> On Fri, Sep 19, 2014 at 5:33 PM, Rodolfo Ochoa <rodolfo.ochoa@gmail.com>
>> wrote:
>>
>>> the problem is...
>>> the problem is that for the generated code,
>>> thrift maps binary data into strings, and uses the method readString from
>>> the BinaryProtocol
>>> protocol.js:280 TBinaryProtocol.prototype.readString = function() {
>>> instead of using readBinary,
>>> protocol.js:275 TBinaryProtocol.prototype.readBinary = function() {
>>> which would return a buffer object.
>>> readString method in the transport will use buffer(...).toString("utf8")
>>> which will sanitize utf and will replace invalid unicode sequences with the
>>> Unicode Replacement Character.
>>>
>>> *my code:*
>>> my binary field was mapped on ondb_types.js as:
>>> TGetResult.prototype = {};
>>> TGetResult.prototype.read = function(input) {
>>>   input.readStructBegin();
>>>   while (true)
>>>   {
>>>     var ret = input.readFieldBegin();
>>>     var fname = ret.fname;
>>>     var ftype = ret.ftype;
>>>     var fid = ret.fid;
>>>     if (ftype == Thrift.Type.STOP) {
>>>       break;
>>>     }
>>>     switch (fid)
>>>     {
>>>       case 1:
>>>       if (ftype == Thrift.Type.STRUCT) {
>>>         this.currentRow = new ttypes.TRow();
>>>         this.currentRow.read(input);
>>>       } else {
>>>         input.skip(ftype);
>>>       }
>>>       break;
>>>       case 2:
>>> *      if (ftype == Thrift.Type.STRING) {*
>>> *        this.currentRowVersion = input.readString();*
>>>       } else {
>>>         input.skip(ftype);
>>>       }
>>>       break;
>>>       default:
>>>         input.skip(ftype);
>>>     }
>>>     input.readFieldEnd();
>>>   }
>>>   input.readStructEnd();
>>>   return;
>>> };
>>>
>>>
>>> and TGetResult is defined in ondb.thrift file as:
>>> typedef binary TVersion
>>> struct TGetResult
>>> {
>>>     1:TRow currentRow;
>>>     2:TVersion currentRowVersion;
>>> }
>>>
>>> I'm not sure if this is already been reported or if this is already been
>>> fixed on the newest version.
>>>
>>> -Rod.O
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Sep 18, 2014 at 11:23 PM, Randy Abernethy <
>>> randy.abernethy@gmail.com> wrote:
>>>
>>>> Hey Rodolfo,
>>>>
>>>> While I can not say for sure not having seen your code, I would be highly
>>>> suspect of any Thift node lib prior to 0.9.2, which is not yet released.
>>>> Try the current master, it has too many fixes to mention and will likely
>>>> solve your problem. You don't need to change versions on any of the other
>>>> languages. Ping back if you still have trouble.
>>>>
>>>> Best,
>>>> Randy
>>>>
>>>> On Thursday, September 18, 2014, Rodolfo Ochoa <rodolfo.ochoa@gmail.com>
>>>> wrote:
>>>>
>>>> > on node:
>>>> > thrift@0.9.1
>>>> >
>>>> > on java:
>>>> > libthrift-0.9.1.jar
>>>> >
>>>> >
>>>> >
>>>> > On Thu, Sep 18, 2014 at 9:09 PM, Randy Abernethy <
>>>> > randy.abernethy@gmail.com <javascript:;>>
>>>> > wrote:
>>>> >
>>>> > > What version of thrift are you using?
>>>> > > On Sep 18, 2014 7:46 PM, "Rodolfo Ochoa" <rodolfo.ochoa@gmail.com
>>>> > <javascript:;>> wrote:
>>>> > >
>>>> > > > I have a Java server sending info to C/Python/node.js clients,
>>>> > > > I don't have issues with C or Python, only with node.js
>>>> > > > on Java side, I'm sending a byte array, and if I print the
values
>>>> of
>>>> > this
>>>> > > > byte array I got:
>>>> > > > byte codes:  -84 -19 0 5 119 44 0 5 -50 108 0 44 75 73 78
-120
>>>> -103 -59
>>>> > > > -124 3 -92 -121 -98 -104 0 0 0 0 0 0 37 -128 1 3 0 0 0 1 0
0 0 1 0
>>>> 0 0
>>>> > 0
>>>> > > 0
>>>> > > > -78 68 -89
>>>> > > >
>>>> > > > but on node.js side I receive:
>>>> > > > byte codes:  65533 65533 0 5 119 44 0 5 65533 108 0 44 75
73 78
>>>> 65533
>>>> > > 65533
>>>> > > > 324 3 65533 65533 65533 65533 0 0 0 0 0 0 37 65533 1 3 0 0
0 1 0 0
>>>> 0 1
>>>> > 0
>>>> > > 0
>>>> > > > 0 0 0 65533 68 65533
>>>> > > >
>>>> > > > this means that some characters were replaced, why? how?
>>>> > > > how can avoid this behavior?
>>>> > > >
>>>> > > >
>>>> > > >
>>>> > > > --
>>>> > > > *Rod.O*
>>>> > > >
>>>> > >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > *Rod.O*
>>>> >
>>>>
>>>
>>>
>>>
>>> --
>>> *Rod.O*
>>>
>>
>>
>>
>> --
>> *Rod.O*
>>
>
>
>
> --
> *Rod.O*

Mime
View raw message