jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix Schumacher <felix.schumac...@internetallee.de>
Subject Re: Strange behaviour when using proxy , any ideas ?
Date Sat, 15 Dec 2018 13:29:56 GMT


Am 15. Dezember 2018 13:38:37 MEZ schrieb Philippe Mouawad <philippe.mouawad@gmail.com>:
>Great you have a simple reproducer.
>On my side , looking inti HC4 I notice issue triggers when we enter
>proxy
>tunnel mode

Right, but we enter that only on https requests. 

>
>http://httpcomponents.10934.n7.nabble.com/Strange-behaviour-when-using-Proxy-td34776.html
>
>
>did you submit a bug or not yet ?

No. I haven't done that. 

Felix 

>
>Thanks
>On Saturday, December 15, 2018, Felix Schumacher <
>felix.schumacher@internetallee.de> wrote:
>
>>
>> Am 14.12.18 um 17:37 schrieb Philippe Mouawad:
>>
>>> Hi Felix,
>>> Thanks for investigations !
>>> Is it an issue due to our code or a bug in hc4?
>>>
>>
>> It looks like a bug in hc4 to me (or we haven't understood that
>feature :)
>>
>> If you change your test code to use https for the request, you will
>get
>> the same behaviour that we see in our bug report.
>>
>>     @Test
>>     public void checkThatHeadersAreNotHidden() throws Exception {
>>         TrustStrategy trustStrategy = new TrustAllStrategy();
>>         SSLContext sslContext = new SSLContextBuilder()
>>                 .loadTrustMaterial(null, trustStrategy).build();
>>         SSLConnectionSocketFactory socketFactory = new
>> SSLConnectionSocketFactory(
>>                 sslContext);
>>         CloseableHttpClient httpclient = HttpClients.custom()
>>                 .setSSLSocketFactory(socketFactory).build();
>>         try {
>>
>>             HttpHost target = new HttpHost("jmeter.apache.org", 443,
>> "https");
>>             HttpHost proxy = new HttpHost("localhost", 8888, "http");
>>
>>             RequestConfig config =
>RequestConfig.custom().setProxy(proxy)
>>                     .build();
>>             HttpGet request = new HttpGet("/");
>>             request.addHeader("X-sleep", "5");
>>             request.setConfig(config);
>>
>>             HttpContext localContext = new BasicHttpContext();
>>             CloseableHttpResponse response =
>httpclient.execute(target,
>> request,
>>                     localContext);
>>             final HttpRequest httpRequestFromLocalContext =
>(HttpRequest)
>> localContext
>>                     .getAttribute(HttpCoreContext.HTTP_REQUEST);
>>             try {
>> Assert.assertThat(httpRequestFromLocalContext.getRequestLine()
>>                         .getMethod(), CoreMatchers.is("CONNECT"));
>> Assert.assertThat(response.getStatusLine().getStatusCode(),
>>                         CoreMatchers.is(200));
>>                 Assert.assertThat(
>> Arrays.asList(request.getAllHeaders()).toString(),
>>                         CoreMatchers.containsString("X-sleep"));
>>                 Assert.assertThat(
>>                         Arrays.asList(
>> httpRequestFromLocalContext.getAllHeaders())
>>                                 .toString(),
>>                         CoreMatchers
>> .not(CoreMatchers.containsString("X-sleep")));
>>             } finally {
>>                 response.close();
>>             }
>>         } finally {
>>             httpclient.close();
>>         }
>>     }
>>
>> I have add a trust all ssl context and changed the class to be a
>JUnit
>> test case.
>>
>> Regards,
>>
>>  Felix
>>
>>
>> ps: didn’t look deeply yet qt your findings
>>>
>>> Regards
>>>
>>> On Friday, December 14, 2018, Felix Schumacher <
>>> felix.schumacher@internetallee.de> wrote:
>>>
>>> Am 14.12.18 um 13:17 schrieb Felix Schumacher:
>>>>
>>>> Am 13.12.18 um 23:28 schrieb Philippe Mouawad:
>>>>>
>>>>> Hello,
>>>>>>
>>>>>> We have a confirmed bug report :
>>>>>>
>>>>>>      - https://bz.apache.org/bugzilla/show_bug.cgi?id=62852
>>>>>>
>>>>>> It appears that when using proxy for a request, the method
>>>>>> request.getRequestHeaders() called on the request returned
>>>>>> HttpContext#getAttribute(HttpCoreContext.HTTP_REQUEST) returns
>only
>>>>>> part of
>>>>>> the headers.
>>>>>>
>>>>>> While it returns them all if not using proxy.
>>>>>>
>>>>>> The headers (context) gets changed on line 189 in
>>>>> org.apache.http.impl.execchain.ProtcolExec.
>>>>>
>>>>>          final CloseableHttpResponse response =
>>>>> this.requestExecutor.execute(route, request,
>>>>>              context, execAware);
>>>>>          try {
>>>>>              // Run response protocol interceptors
>>>>>              context.setAttribute(HttpCoreContext.HTTP_RESPONSE,
>>>>> response); <-- Here
>>>>>              this.httpProcessor.process(response, context);
>>>>>              return response;
>>>>>          } catch (final RuntimeException ex) {
>>>>>
>>>>> Will debug further.
>>>>>
>>>>> Thread [Thread Group 1-1] (Suspended)
>>>> org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl$5(org.
>>>> apache.http.protocol.HttpRequestExecutor).preProcess(org.
>>>> apache.http.HttpRequest,
>>>> org.apache.http.protocol.HttpProcessor,
>org.apache.http.protocol.HttpC
>>>> ontext)
>>>> line: 163
>>>> org.apache.http.impl.execchain.MainClientExec.createTunnelTo
>>>> Target(org.apache.http.auth.AuthState,
>org.apache.http.HttpClientConn
>>>> ection,
>>>> org.apache.http.conn.routing.HttpRoute,
>org.apache.http.HttpRequest,
>>>> org.apache.http.client.protocol.HttpClientContext) line: 472
>>>> org.apache.http.impl.execchain.MainClientExec.establishRoute
>>>> (org.apache.http.auth.AuthState,
>org.apache.http.HttpClientConnection,
>>>> org.apache.http.conn.routing.HttpRoute,
>org.apache.http.HttpRequest,
>>>> org.apache.http.client.protocol.HttpClientContext) line: 411
>>>> org.apache.http.impl.execchain.MainClientExec.execute(org.
>>>> apache.http.conn.routing.HttpRoute, org.apache.http.client.methods
>>>> .HttpRequestWrapper,
>>>> org.apache.http.client.protocol.HttpClientContext,
>>>> org.apache.http.client.methods.HttpExecutionAware) line: 237
>>>> org.apache.http.impl.execchain.ProtocolExec.execute(org.
>>>> apache.http.conn.routing.HttpRoute, org.apache.http.client.methods
>>>> .HttpRequestWrapper,
>>>> org.apache.http.client.protocol.HttpClientContext,
>>>> org.apache.http.client.methods.HttpExecutionAware) line: 185
>>>> org.apache.http.impl.execchain.RetryExec.execute(org.apache.
>>>> http.conn.routing.HttpRoute,
>>>> org.apache.http.client.methods.HttpRequestWrapper,
>>>> org.apache.http.client.protocol.HttpClientContext,
>>>> org.apache.http.client.methods.HttpExecutionAware) line: 89
>>>> org.apache.http.impl.execchain.RedirectExec.execute(org.
>>>> apache.http.conn.routing.HttpRoute, org.apache.http.client.methods
>>>> .HttpRequestWrapper,
>>>> org.apache.http.client.protocol.HttpClientContext,
>>>> org.apache.http.client.methods.HttpExecutionAware) line: 110
>>>> org.apache.http.impl.client.InternalHttpClient.doExecute(org
>>>> .apache.http.HttpHost,
>>>> org.apache.http.HttpRequest, org.apache.http.protocol.HttpContext)
>line:
>>>> 185
>>>> org.apache.http.impl.client.InternalHttpClient(org.apache.ht
>>>> tp.impl.client.CloseableHttpClient).execute(org.apache.http.
>>>> client.methods.HttpUriRequest,
>>>> org.apache.http.protocol.HttpContext) line: 83
>>>> org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.executeR
>>>> equest(org.apache.http.impl.client.CloseableHttpClient,
>>>> org.apache.http.client.methods.HttpRequestBase,
>>>> org.apache.http.protocol.HttpContext, java.net.URL) line: 840
>>>>
>org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(java.net.URL,
>>>> java.lang.String, boolean, int) line: 576
>>>> org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sam
>>>> ple(java.net.URL,
>>>> java.lang.String, boolean, int) line: 67
>>>> org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy(org
>>>> .apache.jmeter.protocol.http.sampler.HTTPSamplerBase).sample()
>line:
>>>> 1231
>>>> org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy(org
>>>> .apache.jmeter.protocol.http.sampler.HTTPSamplerBase).sampl
>>>> e(org.apache.jmeter.samplers.Entry) line: 1220
>>>> org.apache.jmeter.threads.JMeterThread.doSampling(org.apache
>>>> .jmeter.threads.JMeterContext,
>>>> org.apache.jmeter.samplers.Sampler) line: 622
>>>> org.apache.jmeter.threads.JMeterThread.executeSamplePackage(
>>>> org.apache.jmeter.samplers.Sampler, org.apache.jmeter.control.Tran
>>>> sactionSampler,
>>>> org.apache.jmeter.threads.SamplePackage,
>org.apache.jmeter.threads.JMet
>>>> erContext)
>>>> line: 546
>>>> org.apache.jmeter.threads.JMeterThread.processSampler(org.
>>>> apache.jmeter.samplers.Sampler, org.apache.jmeter.samplers.Sampler,
>>>> org.apache.jmeter.threads.JMeterContext) line: 486
>>>>      org.apache.jmeter.threads.JMeterThread.run() line: 253
>>>>      java.lang.Thread.run() line: 748
>>>>
>>>> It seems that httpclient initiates the replacement of the "correct"
>>>> request. I have managed to revert that behaviour by adding a
>save/restore
>>>> functionality in our REQUEST_EXECUTOR that is implemented in
>>>> HTTPHC4Impl. I
>>>> implemented the method preProcess and saved the old request before
>>>> calling
>>>> the super method and restoring the request after the call.
>>>>
>>>> That looks quite ugly and I have no idea, if it is correct. But it
>works
>>>> for the simple test plan.
>>>>
>>>> Felix
>>>>
>>>>
>>>> Felix
>>>>>
>>>>>
>>>>> See:
>>>>>>
>>>>>>      -
>>>>>> https://github.com/apache/jmeter/blob/trunk/src/protocol/
>>>>>>
>http/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java#L580
>>>>>>
>>>>>>
>>>>>> I tried to reproduce behavior with HTTPClient only but couldn't ,
>so it
>>>>>> seems issue is somewhere in our code possibly due to interceptors
>or
>>>>>> request executor.
>>>>>>
>>>>>> /*
>>>>>>    * ============================================================
>>>>>> ========
>>>>>>    * Licensed to the Apache Software Foundation (ASF) under one
>>>>>>    * or more contributor license agreements.  See the NOTICE file
>>>>>>    * distributed with this work for additional information
>>>>>>    * regarding copyright ownership.  The ASF licenses this file
>>>>>>    * to you under the Apache License, Version 2.0 (the
>>>>>>    * "License"); you may not use this file except in compliance
>>>>>>    * with the License.  You may obtain a copy of the License at
>>>>>>    *
>>>>>>    *   http://www.apache.org/licenses/LICENSE-2.0
>>>>>>    *
>>>>>>    * Unless required by applicable law or agreed to in writing,
>>>>>>    * software distributed under the License is distributed on an
>>>>>>    * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
>>>>>>    * KIND, either express or implied.  See the License for the
>>>>>>    * specific language governing permissions and limitations
>>>>>>    * under the License.
>>>>>>    * ============================================================
>>>>>> ========
>>>>>>    *
>>>>>>    * This software consists of voluntary contributions made by
>many
>>>>>>    * individuals on behalf of the Apache Software Foundation. For
>more
>>>>>>    * information on the Apache Software Foundation, please see
>>>>>>    * <http://www.apache.org/>.
>>>>>>    *
>>>>>>    */
>>>>>>
>>>>>> package org.apache.jmeter.protocol.http.proxy;
>>>>>>
>>>>>> import java.util.Arrays;
>>>>>>
>>>>>> import org.apache.http.HttpHost;
>>>>>> import org.apache.http.HttpRequest;
>>>>>> import org.apache.http.client.config.RequestConfig;
>>>>>> import org.apache.http.client.methods.CloseableHttpResponse;
>>>>>> import org.apache.http.client.methods.HttpGet;
>>>>>> import org.apache.http.impl.client.CloseableHttpClient;
>>>>>> import org.apache.http.impl.client.HttpClients;
>>>>>> import org.apache.http.protocol.BasicHttpContext;
>>>>>> import org.apache.http.protocol.HttpContext;
>>>>>> import org.apache.http.protocol.HttpCoreContext;
>>>>>>
>>>>>> /**
>>>>>>    * How to send a request via proxy.
>>>>>>    *
>>>>>>    * @since 4.0
>>>>>>    */
>>>>>> public class TestProxy {
>>>>>>
>>>>>>       public static void main(String[] args)throws Exception {
>>>>>>           CloseableHttpClient httpclient =
>HttpClients.createDefault();
>>>>>>           try {
>>>>>>
>>>>>>               HttpHost target = new HttpHost("jmeter.apache.org",
>80,
>>>>>> "http");
>>>>>>               HttpHost proxy = new HttpHost("localhost", 8888,
>"http");
>>>>>>
>>>>>>               RequestConfig config = RequestConfig.custom()
>>>>>>                       .setProxy(proxy)
>>>>>>                       .build();
>>>>>>               HttpGet request = new HttpGet("/");
>>>>>>               request.addHeader("X-sleep", "5");
>>>>>>               request.setConfig(config);
>>>>>>
>>>>>>               System.out.println("Executing request " +
>>>>>> request.getRequestLine() + " to " + target + " via " + proxy);
>>>>>>               HttpContext localContext = new BasicHttpContext();
>>>>>>               CloseableHttpResponse response =
>>>>>> httpclient.execute(target,
>>>>>> request, localContext);
>>>>>>               final HttpRequest httpRequestFromLocalContext =
>>>>>> (HttpRequest)
>>>>>> localContext.getAttribute(HttpCoreContext.HTTP_REQUEST);
>>>>>>               try {
>>>>>>
>>>>>> System.out.println("----------------------------------------");
>>>>>>                   System.out.println(response.getStatusLine());
>>>>>> System.out.println(Arrays.asList(request.getAllHeaders()));
>>>>>>
>>>>>> System.out.println(Arrays.asList(httpRequestFromLocalContext
>>>>>> .getAllHeaders()));
>>>>>>
>>>>>> System.out.println(Arrays.asList(response.getAllHeaders()));
>>>>>>
>>>>>> //System.out.println(EntityUtils.toString(response.getEntity()));
>>>>>>               } finally {
>>>>>>                   response.close();
>>>>>>               }
>>>>>>           } finally {
>>>>>>               httpclient.close();
>>>>>>           }
>>>>>>       }
>>>>>>
>>>>>> }
>>>>>>
>>>>>> OUTPUT:
>>>>>> ------------------------------------------------------------
>>>>>> ----------------------------------------
>>>>>> Executing request GET / HTTP/1.1 to http://jmeter.apache.org:80
>via
>>>>>> http://localhost:8888
>>>>>> ----------------------------------------
>>>>>> HTTP/1.1 200 OK
>>>>>> [X-sleep: 5]
>>>>>> [X-sleep: 5, Host: jmeter.apache.org:80, Proxy-Connection:
>Keep-Alive,
>>>>>> User-Agent: Apache-HttpClient/4.5.6 (Java/1.8.0_161),
>Accept-Encoding:
>>>>>> gzip,deflate]
>>>>>> [Date: Thu, 13 Dec 2018 22:23:23 GMT, Server: Apache/2.4.18
>(Ubuntu),
>>>>>> Last-Modified: Sun, 09 Sep 2018 15:25:54 GMT, ETag:
>>>>>> "3159-57571db50c67a-gzip", Accept-Ranges: bytes, Vary:
>Accept-Encoding,
>>>>>> Keep-Alive: timeout=30, max=100, Connection: Keep-Alive,
>Content-Type:
>>>>>> text/html, Content-Length: 12633]
>>>>>> ------------------------------------------------------------
>>>>>> ----------------------------------------
>>>>>>
>>>>>>
>>>>>> Regards
>>>>>> Philippe M.
>>>>>> <https://www.openstreetmap.org/#map=18/50.69454/3.16455>
>>>>>>
>>>>>>
>>>>>>

Mime
View raw message