Hi YuMing,
it looks 0.94 is immune to this problem. if read 0 bytes, just returns
readAndProcess:
count = channelRead(channel, data);
if (data.remaining() == 0) {
dataLengthBuffer.clear();
data.flip();
if (headerRead) {
processData(data.array());
data = null;
return count;
}
processHeader();
headerRead = true;
data = null;
continue;
}
return count;
}
On Tue, Jul 15, 2014 at 11:06 AM, Qiang Tian <tianq01@gmail.com> wrote:
> Hi, below is more details.
>
> the read0 stacktrace you see means reader wants to read something from
> client RPC call. in Andrew's test it shows it is in reading RPC request
> data (reasonable. since other meta data size is quite small). although
> client follows request-receive style, when multiple clients share the
> connection(the default case), the synchronization window when writing to
> the same channel is quite small. if those request data have big size, there
> might be a sudden rush to the transportation layer..might causing RPC
> server could not receive the data in time due to congestion control,
> without hbase11277, the reader get 0 byte read again and again...
>
> with hbase11277 the problem should be resolved - we get back to complete
> non-blocking IO, but it is still worth investigation non-shared connection
> under high workload(hbase11306).
>
>
>
> On Tue, Jul 15, 2014 at 10:44 AM, Rural Hunter <ruralhunter@gmail.com>
> wrote:
>
>> No. My issue is definitely a server side issue. When the problem happens
>> I'm not able to telnet the data port of RS.
>>
>> δΊ 2014/7/15 10:24, Qiang Tian ει:
>>
>> not sure if you could also try using different connections for your
>>> client
>>> program. see the related issue HBASE-11306. I guess sharing connection
>>> is
>>> a contributing factor for this issue.
>>>
>>>
>>>
>>
>
|