Unable to send requests with literal '+' character in URI query part

java

#1

Hi everyone,

I’ve already filed an issue on GitHub at https://github.com/playframework/play-ws/issues/287 because I think it’s a bug in Play WS (or the underlying HTTP library), but maybe I’m just “holding it wrong”. Sorry for the noise. :wink:

I want to send a request to a remote HTTP service which has some strict encoding rules.

For example, HTTP requests to http://example.com/path?query+string are working, but requests to http://example.com/path?query%2Bstring are not working.

Unfortunately, Play WS always encodes the URL as http://example.com/path?query%2Bstring without the option to skip unnecessary encoding or to provide a raw query string.

Play WS Version

Play WS 2.6.20

$ sbt dependencyList|grep -E 'play-(ahc|ws)|asynchttp'
[info] com.typesafe.play:play-ahc-ws-standalone_2.12:1.1.10
[info] com.typesafe.play:play-ahc-ws_2.12:2.6.20
[info] com.typesafe.play:play-ws-standalone-json_2.12:1.1.10
[info] com.typesafe.play:play-ws-standalone-xml_2.12:1.1.10
[info] com.typesafe.play:play-ws-standalone_2.12:1.1.10
[info] com.typesafe.play:play-ws_2.12:2.6.20
[info] com.typesafe.play:shaded-asynchttpclient:1.1.10

API (Scala / Java / Neither / Both)

Java

Operating System

MacOS 10.13.6

$ uname -rspv
Darwin 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 i386

JDK

java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)

Expected Behavior

  1. Create WSRequest with valid URL via WSClient#url(String), for example http://example.com/path?query+string.
  2. Execute HTTP request via WSRequest#get().
  3. HTTP request is being sent to the URL provided to WSRequest (http://example.com/path?query+string).

Actual Behavior

  1. Create WSRequest with valid URL via WSClient#url(String), for example http://example.com/path?query+string.
  2. Execute HTTP request via WSRequest#get().
  3. HTTP request is being sent to an incorrectly (or too eagerly) encoded URL (http://example.com/path?query%2Bstring).

Reproducible Test Case

import com.github.tomakehurst.wiremock.junit.WireMockRule;
import org.junit.Rule;
import org.junit.Test;
import play.libs.ws.WSClient;
import play.libs.ws.WSRequest;
import play.libs.ws.WSResponse;
import play.libs.ws.ahc.AhcWSClient;
import play.shaded.ahc.org.asynchttpclient.DefaultAsyncHttpClient;

import static com.github.tomakehurst.wiremock.client.WireMock.*;
import static org.junit.Assert.assertEquals;

public class WSRequestQueryParametersTest {
    @Rule
    public WireMockRule wireMockRule = new WireMockRule();

    @Test
    public void uriEncode() throws Exception {
        WSClient wsClient = new AhcWSClient(new DefaultAsyncHttpClient(), null);
        String requestUrl = wireMockRule.url("/path") + "?the+plus+must+remain";
        stubFor(get(urlEqualTo("/path?the+plus+must+remain")).willReturn(aResponse()
                .withStatus(200)
                .withBody("OK")));

        WSRequest wsRequest = wsClient.url(requestUrl);
        String url = wsRequest.getUrl();
        assertEquals("Request URL and WSRequest.Url should be identical", requestUrl, url);

        WSResponse wsResponse = wsRequest.get().toCompletableFuture().get();
        assertEquals(200, wsResponse.getStatus());

        verify(getRequestedFor(urlMatching("/path")).withQueryParam("the+plus+must+remain", equalTo("")));
    }
}
2018-10-26 15:31:57.070 +0200 [] [INFO] Logging initialized @1786ms to org.eclipse.jetty.util.log.Slf4jLog  
2018-10-26 15:31:57.248 +0200 [] [INFO] jetty-9.4.12.v20180830; built: 2018-08-30T13:59:14.071Z; git: 27208684755d94a92186989f695db2d7b21ebc51; jvm 1.8.0_181-b13  
2018-10-26 15:31:57.278 +0200 [] [INFO] Started o.e.j.s.ServletContextHandler@5b218417{/__admin,null,AVAILABLE}  
2018-10-26 15:31:57.281 +0200 [] [INFO] Started o.e.j.s.ServletContextHandler@413d1baf{/,null,AVAILABLE}  
2018-10-26 15:31:57.325 +0200 [] [INFO] Started NetworkTrafficServerConnector@31995969{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}  
2018-10-26 15:31:57.326 +0200 [] [INFO] Started @2045ms  
2018-10-26 15:31:58.032 +0200 [] [INFO] RequestHandlerClass from context returned com.github.tomakehurst.wiremock.http.AdminRequestHandler. Normalized mapped under returned 'null'  
2018-10-26 15:31:58.379 +0200 [] [INFO] RequestHandlerClass from context returned com.github.tomakehurst.wiremock.http.StubRequestHandler. Normalized mapped under returned 'null'  
2018-10-26 15:31:58.395 +0200 [] [ERROR] 
                                               Request was not matched
                                               =======================

-----------------------------------------------------------------------------------------------------------------------
| Closest stub                                             | Request                                                  |
-----------------------------------------------------------------------------------------------------------------------
                                                           |
GET                                                        | GET
/path?the+plus+must+remain                                 | /path?the%2Bplus%2Bmust%2Bremain                    <<<<< URL does not match
                                                           |
                                                           |
-----------------------------------------------------------------------------------------------------------------------
  
2018-10-26 15:31:58.575 +0200 [] [INFO] Stopped NetworkTrafficServerConnector@31995969{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}  
2018-10-26 15:31:58.577 +0200 [] [INFO] Stopped o.e.j.s.ServletContextHandler@413d1baf{/,null,UNAVAILABLE}  
2018-10-26 15:31:58.577 +0200 [] [INFO] Stopped o.e.j.s.ServletContextHandler@5b218417{/__admin,null,UNAVAILABLE}  

java.lang.AssertionError: 
Expected :200
Actual   :404

#2

The same issue still exists on Play WS 2.0.0-RC1.

I’ve created a repository with reproducible test cases for Play WS 1.1.12 and 2.0.0-RC1 at

Does anyone have an idea how to fix this or has a workaround for the issue?