hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [hadoop] bharatviswa504 commented on issue #549: HDDS-1213. Support plain text S3 MPU initialization request
Date Tue, 05 Mar 2019 22:47:19 GMT
bharatviswa504 commented on issue #549: HDDS-1213. Support plain text S3 MPU initialization
request
URL: https://github.com/apache/hadoop/pull/549#issuecomment-469889963
 
 
   Over all patch LGTM.
   Thank You @elek  for offline discussion.
   Yes, I see that when file size is large, cp is using multipart upload request.
   
   Few minor comments:
   1. Remove unncessary space indentation changes
   2. Removed platform.system(), as we don't need it now.
   
   I see below error when running test manually with docker ozones3 cluster
   
   ```
   s3g_1           | 2019-03-05 22:37:40 WARN  HttpChannel:499 - //localhost:9878/b12345/mm11?partNumber=3&uploadId=d51657f3-3276-49d0-977a-af9f4bab94bc-19533321172195
   s3g_1           | javax.servlet.ServletException: javax.servlet.ServletException: org.glassfish.jersey.server.ContainerException:
java.lang.OutOfMemoryError: Java heap space
   s3g_1           | 	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:139)
   s3g_1           | 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   s3g_1           | 	at org.eclipse.jetty.server.Server.handle(Server.java:539)
   s3g_1           | 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
   s3g_1           | 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   s3g_1           | 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
   s3g_1           | 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
   s3g_1           | 	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   s3g_1           | 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   s3g_1           | 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   s3g_1           | 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   s3g_1           | 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   s3g_1           | 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   s3g_1           | 	at java.lang.Thread.run(Thread.java:748)
   s3g_1           | Caused by: javax.servlet.ServletException: org.glassfish.jersey.server.ContainerException:
java.lang.OutOfMemoryError: Java heap space
   s3g_1           | 	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:432)
   s3g_1           | 	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370)
   s3g_1           | 	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389)
   s3g_1           | 	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342)
   s3g_1           | 	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229)
   s3g_1           | 	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:840)
   s3g_1           | 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
   s3g_1           | 	at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1609)
   s3g_1           | 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   s3g_1           | 	at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
   s3g_1           | 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   s3g_1           | 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   s3g_1           | 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
   s3g_1           | 	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
   s3g_1           | 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
   s3g_1           | 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   s3g_1           | 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   s3g_1           | 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   s3g_1           | 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   s3g_1           | 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   s3g_1           | 	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
   s3g_1           | 	... 13 more
   s3g_1           | Caused by: org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError:
Java heap space
   ```
   
   I see when allocating a buffer, I am getting this error. I see by default we allocate 2
64mb buffer for all buffer lists. Any idea how to resolve this?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message