thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randy Abernethy <randy.aberne...@gmail.com>
Subject Re: When will TProtocolUtil.skip() be called
Date Tue, 13 Jan 2015 03:31:36 GMT
Hello,

It is hard to say why skip might be consuming so much cpu. A code defect is
certainly possible. I haven't looked at 0.7 so can not say more.

On the upgrade front. Apache Thrift is not yet at 1.0 release and changes
from version to version could cause incompatibilities depending on the
languages involved. I would test any one sided upgrades carefully before
committing to the change.

Best,
Randy

On Mon, Jan 12, 2015 at 7:02 PM, Mingmin Liu <diveintotomorrow@gmail.com>
wrote:

> Hi Randy,
>
> Thanks for your reply.
>
> In my case, the server is guaranteed to be using the latest IDL, and we
> have not deleted any field when evolving our IDL.
>
> So may I say that the TProtocolUtil.skip is called because it is trying
> to deserialize some broken or malicious binaries?
>
> I also notice that the server spend a lot of time(7% of cpu time) calling
> TProtocolUtil.skip,  is that an expected behaviour?
>
>
>
> Also, is it safe for us to update the server to 0.9.2, while leave the
> clients using 0.7.0?
>
> Thanks
>
> On Tue Jan 13 2015 at 2:09:34 AM Randy Abernethy <ra@apache.org> wrote:
>
>> Hello Mingmin Liu,
>>
>> There is at least one other case for skip.
>>
>> 3. new server sees data from old client and old client is still
>> transmitting a field which has been deleted in new server IDL.
>>
>> Also I would note that Apache Thrift 0.7.0 is several years old, the
>> current release is 0.9.2 and many improvements and subtle changes have
>> taken place since 0.7.0.
>>
>> Best,
>> Randy
>>
>> On Mon, Jan 12, 2015 at 12:55 AM, Mingmin Liu
>> <diveintotomorrow@gmail.com> wrote:
>> > I am using thrift 0.7.0 on my server, and recently find sometimes my
>> server
>> > busy calling TProtocolUtil.skip().
>> >
>> > stack trace:
>> >
>> > "New I/O  worker #75" daemon prio=10 tid=0x00007f7804892000
>> > nid=0x51601 runnable [0x00007f76129e7000]
>> >    java.lang.Thread.State: RUNNABLE
>> >     at org.apache.thrift.protocol.TProtocolUtil.skip(TProtocolUtil.
>> java:122)
>> >     at org.apache.thrift.protocol.TProtocolUtil.skip(TProtocolUtil.
>> java:60)
>> >     at com.My_Thrift_Object.read(My_Thrift_Object.java:958)
>> >     at org.apache.thrift.TDeserializer.deserialize(TDeserializer.
>> java:69)
>> >
>> > this happens when I try to deserialize the binary content from client.
>> >
>> > Is it true that TProtocolUtil .skip() will be called only  when
>> > My_Thrift_Object see some field that it doesn't know ?
>> >
>> > if yes, when will this happen?
>> >
>> > I can come up with 2 conditions:
>> >
>> > 1. old server see data from new client and the new client has defined
>> some
>> > new field in thrift.
>> > 2. the binary content is corrupted or someone maliciously build up it.
>> >
>> > is my understanding right?
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message