airavata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcus Christie (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRAVATA-2692) Django: hitting a 15 second timeout of some sort with large file uploads
Date Tue, 08 Jan 2019 23:26:00 GMT

    [ https://issues.apache.org/jira/browse/AIRAVATA-2692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737635#comment-16737635
] 

Marcus Christie commented on AIRAVATA-2692:
-------------------------------------------

Some other things I tried:
* Tried setting Apache TimeOut to 600 in virtualhost config
* Tried setting LogLevel to debug
This does show lines like:
{noformat}
[Tue Jan 08 22:43:29.397499 2019] [ssl:debug] [pid 21855] ssl_engine_io.c(993): [client 149.160.145.174:49379]
AH02001: Connection closed to child 8 with standard shutdown (server django.seagrid.org:443)
{noformat}
but I think those are just normal things to log when the connection is closed. I see those
after ever connection even without a timeout.
* This sounds exactly like my problem: https://serverfault.com/q/871556
* Tried disabling RequestReadTimeout
{noformat}
    RequestReadTimeout header=0 body=0
{noformat}
* Tried setting {{ProxyTimeout 60}} in virtualhost config
* 

> Django: hitting a 15 second timeout of some sort with large file uploads
> ------------------------------------------------------------------------
>
>                 Key: AIRAVATA-2692
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-2692
>             Project: Airavata
>          Issue Type: Bug
>            Reporter: Marcus Christie
>            Assignee: Marcus Christie
>            Priority: Major
>
> For some reason it fails to get the project when doing a file upload
> {noformat}
> [Fri Feb 23 03:46:37.783086 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
[2018-02-23 03:46:37,782 django_airavata.apps.api.views:652 ERROR] Failed to upload file
> [Fri Feb 23 03:46:37.783108 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
Traceback (most recent call last):
> [Fri Feb 23 03:46:37.783113 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
  File "/var/www/portals/django-seagrid/django-airavata-gateway/django_airavata/apps/api/views.py",
line 642, in upload_input_file
> [Fri Feb 23 03:46:37.783118 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
    request.authz_token, project_id)
> [Fri Feb 23 03:46:37.783122 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
  File "/var/www/portals/django-seagrid/django-airavata-gateway/airavata/api/Airavata.py",
line 4733, in getProject
> [Fri Feb 23 03:46:37.783126 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
    return self.recv_getProject()
> [Fri Feb 23 03:46:37.783130 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
  File "/var/www/portals/django-seagrid/django-airavata-gateway/airavata/api/Airavata.py",
line 4746, in recv_getProject
> [Fri Feb 23 03:46:37.783134 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
    (fname, mtype, rseqid) = iprot.readMessageBegin()
> [Fri Feb 23 03:46:37.783138 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
  File "/var/www/portals/django-seagrid/venv/lib64/python3.4/site-packages/thrift/protocol/TBinaryProtocol.py",
line 134, in readMessageBegin
> [Fri Feb 23 03:46:37.783152 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
    sz = self.readI32()
> [Fri Feb 23 03:46:37.783185 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
  File "/var/www/portals/django-seagrid/venv/lib64/python3.4/site-packages/thrift/protocol/TBinaryProtocol.py",
line 217, in readI32
> [Fri Feb 23 03:46:37.783190 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
    buff = self.trans.readAll(4)
> [Fri Feb 23 03:46:37.783194 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
  File "/var/www/portals/django-seagrid/venv/lib64/python3.4/site-packages/thrift/transport/TTransport.py",
line 60, in readAll
> [Fri Feb 23 03:46:37.783198 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
    chunk = self.read(sz - have)
> [Fri Feb 23 03:46:37.783202 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
  File "/var/www/portals/django-seagrid/venv/lib64/python3.4/site-packages/thrift/transport/TTransport.py",
line 161, in read
> [Fri Feb 23 03:46:37.783206 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
    self.__rbuf = BufferIO(self.__trans.read(max(sz, self.__rbuf_size)))
> [Fri Feb 23 03:46:37.783210 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
  File "/var/www/portals/django-seagrid/venv/lib64/python3.4/site-packages/thrift/transport/TSocket.py",
line 132, in read
> [Fri Feb 23 03:46:37.783239 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
    message='TSocket read 0 bytes')
> [Fri Feb 23 03:46:37.783251 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
thrift.transport.TTransport.TTransportException: TSocket read 0 bytes
> [Fri Feb 23 03:46:37.783263 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]

> {noformat}
> This is hitting a 15 second timeout. Not sure where it is at.
> Maybe in mod_wsgi?
> https://groups.google.com/forum/#!topic/modwsgi/zuhFREjOE8M
> http://modwsgi.readthedocs.io/en/master/configuration-directives/WSGIDaemonProcess.html?highlight=timeout
> I know it is a 15 second timeout thing and not related to the size of the file. If I
put {{time.sleep(15)}} in my Django view handler and upload a very small file it triggers
the same Thrift error. Weird that it causes Thrift to generate this error.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message