tinkerpop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "MichaelZ (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TINKERPOP-2268) Prevent Connection Failure from Hanging
Date Thu, 08 Aug 2019 19:42:00 GMT

    [ https://issues.apache.org/jira/browse/TINKERPOP-2268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16903268#comment-16903268
] 

MichaelZ commented on TINKERPOP-2268:
-------------------------------------

# my understanding of the concept here is that such socket code tries to optimize connections
until they timeout for lack of activity, so client-side.
 # yes, a workaround that allows me to leverage your client and stephen's gremlin insights. 
first i just checked if connected, else re-instantiate client.  next more elegant solution,
i just checked if first connection connected (because the code always checks pool in that
order), and if not, then re-instantiate pool. this approach is still working flawlessly in
a PROD app after several weeks.
 # the more elegant solution may be that when we find a connection ! IsOpen, then re-connect
that connection and then use it for that request.  but that is more complicated because we
have to allow for other concurrent requests against that client.  what say you?
 # I did have to deal with the issue mentioned (but don't think it's related)  in https://issues.apache.org/jira/browse/TINKERPOP-2192 in
that i would check if the message was null before trying to Parse() it.  in the function
Connection.TryParseResponseMessage i added:
 # 
{code:java}
//to reduce trying to process nulls
 if (receivedMsg == null)
 {
 return;
 }{code}
i am not sure why sometimes the message is null...could be messaging threading.
 # i will try to set up a testbed to get the exception.  it could be that the code is slow
to error out when calling submit on a dead connection.  as you know the code will throw a throw
new ConnectionPoolBusyException(_poolSize, _maxInProcessPerConnection) if no connection has
less than max requests and an open connection, which may conflict with RemoveConnectionFromPool,
which is calling DefinitelyDestroyConnection.  i find that if i lose the first one, it's
just easier to refresh the pool.  i haven't load tested this, but it seems that if a connection
times out for lack of activity, the others are not far behind, and if there is activity, all
connections will be ok.  what connection pools allow for is scaling.  a more elegant solution
would improve scaling, but may complicate the code to handle the edge case.  i don't know
the trade-offs.
 # looking forward to helping you...help us!

> Prevent Connection Failure from Hanging
> ---------------------------------------
>
>                 Key: TINKERPOP-2268
>                 URL: https://issues.apache.org/jira/browse/TINKERPOP-2268
>             Project: TinkerPop
>          Issue Type: Improvement
>          Components: dotnet, driver
>         Environment: .Net Core
>            Reporter: MichaelZ
>            Priority: Major
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> When a consumer of the Gremlin.Net client calls to execute a Gremlin query, i.e. "SubmitAsync,"
and there is no valid connection, there will be a costly timeout error. I have experienced
30 to 90 second timeouts.
> I was on vacation, so I didn't do this earlier, but I have written a little patch that
will refresh the connection pool when there is no valid connection, and it works flawlessly. 
This is quick code, and there is a more elegant solution, but what I did is to check IsOpen
on the first connection snapshot, and create a new pool if it was stale.  Here is is the
code on the GremlinClient object:
> {code}
> private ConnectionPool _connectionPool; //{color:#ff0000}used to be readonly{color}
> // member variables
>  private readonly GremlinServer _gremlinServer = null;
>  private readonly GraphSONReader _graphSONReader = null;
>  private readonly GraphSONWriter _graphSONWriter = null;
>  private readonly string _mimeType = null;
> private readonly ConnectionPoolSettings _connectionPoolSettings = null;
> private readonly Action<ClientWebSocketOptions> _webSocketConfiguration = null;
>  //
> public GremlinClient(GremlinServer gremlinServer, GraphSONReader graphSONReader = null,
>  GraphSONWriter graphSONWriter = null, string mimeType = null,
>  ConnectionPoolSettings connectionPoolSettings = null,
>  Action<ClientWebSocketOptions> webSocketConfiguration = null)
>  {
>  //
>  _gremlinServer = gremlinServer;
>  _graphSONReader = graphSONReader;
>  _graphSONWriter = graphSONWriter;
>  _mimeType = mimeType;
>  _connectionPoolSettings = connectionPoolSettings;
>  _webSocketConfiguration = webSocketConfiguration;
>  //
>  {color:#ff0000}NewConnectionPool(){color};
>  }
> private void NewConnectionPool()
> {
> var reader = _graphSONReader ?? new GraphSON3Reader();
> var writer = _graphSONWriter ?? new GraphSON3Writer();
> var connectionFactory = new ConnectionFactory(_gremlinServer, reader, writer, _mimeType
?? DefaultMimeType, _webSocketConfiguration);
> _connectionPool = new ConnectionPool(connectionFactory, _connectionPoolSettings ?? new
ConnectionPoolSettings());
> }
> /// <summary>
>  /// Provides whether the first available connection snapshot in pool is still open.
>  /// </summary>
>  {color:#ff0000}private{color} bool HasOpenConnection => (bool)_connectionPool?.FirstConnectionSnapshot?.IsOpen;
> /// <inheritdoc />
>  public async Task<ResultSet<T>> SubmitAsync<T>(RequestMessage requestMessage)
>  {
>  if (!HasOpenConnection)
>  {
>  Debug.WriteLine("=====================================");
>  Debug.WriteLine("new connection pool");
> {color:#ff0000}NewConnectionPool(){color};
>  }
> using (var connection = await _connectionPool.GetAvailableConnectionAsync().ConfigureAwait(false))
> { return await connection.SubmitAsync<T>(requestMessage).ConfigureAwait(false);
}
> }
> {code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Mime
View raw message