drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [drill] cgivre commented on a change in pull request #2270: DRILL-7970: Add URL Parameters to HTTP Plugin
Date Fri, 06 Aug 2021 20:15:04 GMT

cgivre commented on a change in pull request #2270:
URL: https://github.com/apache/drill/pull/2270#discussion_r684481712



##########
File path: contrib/storage-http/src/main/java/org/apache/drill/exec/store/http/util/SimpleHttp.java
##########
@@ -322,6 +336,99 @@ private void setupCache(Builder builder) {
     return formBodyBuilder;
   }
 
+  /**
+   * Returns the URL-decoded URL
+   *
+   * @return Returns the URL-decoded URL
+   */
+  public static String decodedURL(HttpUrl url) {
+    try {
+      return URLDecoder.decode(url.toString(), "UTF-8");
+    } catch (UnsupportedEncodingException e) {
+      return url.toString();
+    }
+  }
+
+  /**
+   * Returns true if the url has url parameters, as indicated by the presence of
+   * {param} in a url.
+   *
+   * @return True if there are URL params, false if not
+   */
+  public static boolean hasURLParameters(HttpUrl url) {
+    String decodedUrl = SimpleHttp.decodedURL(url);
+    Matcher matcher = URL_PARAM_REGEX.matcher(decodedUrl);
+    return matcher.find();
+  }
+
+  /**
+   * APIs sometimes are structured with parameters in the URL itself.  For instance, to request
a list of
+   * an organization's repositories in github, the URL is: https://api.github.com/orgs/{org}/repos,
where
+   * you can replace the org with the actual organization name.
+   *
+   * @return A list of URL parameters enclosed by curly braces.
+   */
+  public static List<String> getURLParameters(HttpUrl url) {
+    List<String> parameters = new ArrayList<>();
+    String decodedURL;
+    try {
+      decodedURL = URLDecoder.decode(url.toString(), "UTF-8");
+    } catch (UnsupportedEncodingException e) {
+      return null;
+    }
+    Matcher matcher = URL_PARAM_REGEX.matcher(decodedURL);
+    String param;
+    while (matcher.find()) {
+      param = matcher.group();
+      param = param.replace("{", "");
+      param = param.replace("}", "");
+      parameters.add(param);
+    }
+    return parameters;
+  }
+
+  /**
+   * Used for APIs which have parameters in the URL.  This function maps the filters pushed
down
+   * from the query into the URL.  For example the API: github.com/orgs/{org}/repos requires
a user to
+   * specify an organization and replace {org} with an actual organization.  The filter is
passed down from
+   * the query.
+   *
+   * Note that if a URL contains URL parameters and one is not provided in the filters, Drill
will throw
+   * a UserException.
+   *
+   * @param url The HttpUrl containing URL Parameters
+   * @param filters:  A HashMap of filters
+   * @return A string of the URL with the URL parameters replaced by filter values
+   */
+  public static String mapURLParameters(HttpUrl url, Map<String, String> filters) {
+    if (! hasURLParameters(url)) {
+      return url.toString();
+    }
+
+    List<String> params = SimpleHttp.getURLParameters(url);
+    String tempUrl = SimpleHttp.decodedURL(url);
+    for (String param : params) {
+      if (filters == null) {
+        throw UserException
+          .parseError()
+          .message("API Query with URL Parameters must be populated. Parameter " + param
+ " must be included in WHERE clause.")
+          .build(logger);
+      }
+
+      String value = filters.get(param);

Review comment:
       I converted this to use  a `CaseInsensitiveMap`.   Thus... it is now case insensitive.
  Added unit test for the URL mapper.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message