thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randy Abernethy <randy.aberne...@gmail.com>
Subject Re: JMeter load test Thrift using multi threads got TTransportException: Cannot write to null outputStream
Date Thu, 07 Apr 2016 12:50:24 GMT
Hey,

So if the same machine can do 3,000 Java threads/sockets on one OS and not
on another we are almost definitely running into system based limits. The
stack trace makes me believe that you are running out of sockets (allowed
or resources required to create). What edition of Windows 7 are you
running? In general desktop OSes use limits designed to protect an end user
rather than limit designed to support a server needing many client
connections.

Here is a good discussion of the Windows settings you might need to look at
tweaking:
http://smallvoid.com/article/winnt-tcpip-max-limit.html

Or you could just try running the load test on a modern server OS (probably
more like what you will deploy the server on anyway, yes?).

Good luck!
Randy


On Wed, Apr 6, 2016 at 7:33 PM, Tenghuan He <tenghuanhe@gmail.com> wrote:

> Hi Randy
>
> Thanks for you replay
>
> 1. The environment is Windows 7 64 bit 4G memory 4 cores
> 2. The server is running on the same machine with JMeter,
> 3. I used this command on cygwin on my windows , the result is '256', does
> that make sense?
> 4. Both the client and server are written in Java
>
> I test using the same code and .jmx files in VMWare Ubuntu virtual machine
> with 2G memory, raise the threads to 3000 ramp up in 1 seconds and it works
> well
> when to 5000 threads, I got three kind of errors:
> org.apache.thrift.transport.TTransportException: java.net.SocketException:
> Connection reset
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection timed out
> org.apache.thrift.transport.TTransportException: Cannot write to null
> outputStream
>
>
>
> On Thu, Apr 7, 2016 at 6:15 AM, Randy Abernethy <ra@apache.org> wrote:
>
> > Hello,
> >
> > Sounds like your system is running out of resources due to shortage or
> > limits.
> > You could be out of memory (virt mode or phys), out of file descriptors,
> > etc.
> > Some questions:
> >
> >  - How is the machine you are using JMeter on sized (mem/swap in
> > particular)?
> >  - Where is the server running (if its on the machine you are running
> > JMeter on
> > two sockets are being used for every client thread)?
> >  - What does "ulimit -n" tell you (the max file descriptors per proc on
> > most
> > linux systems is 1024)?
> >  - I can see the client is Java, what is the server written in?
> >
> > -Randy
> >
> >
> > On Wed, Apr 6, 2016 at 11:01 AM, Tenghuan He <tenghuanhe@gmail.com>
> wrote:
> >
> > > Hi all,
> > >
> > > I am using JMeter to load test my Thrift server.
> > > TThreadPoolServer was used
> > >
> > > In JMeter when I set the number of threads to more than 2000 (where
> each
> > > thread will create an client with its own TSocket and TProtocol) and
> the
> > > ramp up time in 1 second. After a while the client got the following
> > error
> > >
> > > org.apache.thrift.transport.TTransportException: Cannot write to null
> > > outputStre
> > > am
> > >         at
> > > org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTranspo
> > > rt.java:142)
> > >         at
> > > org.apache.thrift.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.j
> > > ava:178)
> > >         at
> > > org.apache.thrift.protocol.TBinaryProtocol.writeMessageBegin(TBinaryP
> > > rotocol.java:106)
> > >         at
> > > org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:70)
> > >         at
> > > org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
> > >         at
> > > com.corp.tsdb.common.rpc.RPCQueryProcessor$Client.send_getDataPoint(R
> > > PCQueryProcessor.java:130)
> > >         at
> > > com.corp.tsdb.common.rpc.RPCQueryProcessor$Client.getDataPoint(RPCQue
> > > ryProcessor.java:123)
> > >         at
> > > com.corp.tsdb.common.rpc.RPCQueryProcessorClient.getDataPoint(RPCQuer
> > > yProcessorClient.java:62)
> > >         at
> > > com.corp.tsdb.common.rpc.loadtest.ThriftSampler.runTest(ThriftSampler
> > > .java:45)
> > >         at
> > > org.apache.jmeter.protocol.java.sampler.JavaSampler.sample(JavaSample
> > > r.java:191)
> > >         at
> > > org.apache.jmeter.threads.JMeterThread.process_sampler(JMeterThread.j
> > > ava:434)
> > >         at
> > > org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:261)
> > >         at java.lang.Thread.run(Thread.java:745)
> > >
> > > Could anyone help me?
> > > Thanks
> > >
> > > Tenghuan
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message