nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lewismc <...@git.apache.org>
Subject [GitHub] nutch pull request #141: NUTCH-2300 Fetcher to optionally save robots.txt
Date Fri, 19 Aug 2016 21:05:27 GMT
Github user lewismc commented on a diff in the pull request:

    https://github.com/apache/nutch/pull/141#discussion_r75551347
  
    --- Diff: src/plugin/lib-http/src/java/org/apache/nutch/protocol/http/api/HttpBase.java
---
    @@ -563,23 +558,25 @@ protected static void main(HttpBase http, String[] args) throws
Exception {
       protected abstract Response getResponse(URL url, CrawlDatum datum,
           boolean followRedirects) throws ProtocolException, IOException;
     
    -  public BaseRobotRules getRobotRules(Text url, CrawlDatum datum) {
    -    return robots.getRobotRulesSet(this, url);
    +  @Override
    +  public BaseRobotRules getRobotRules(Text url, CrawlDatum datum,
    +      List<Content> robotsTxtContent) {
    +    return robots.getRobotRulesSet(this, url, robotsTxtContent);
       }
       
       /**
        * Transforming a String[] into a HashMap for faster searching
        * @param input String[]
        * @return a new HashMap
        */
    -  private HashMap arrayToMap(String[]input){
    -    if (input==null ||input.length==0) {
    -      return new HashMap();
    +  private HashMap<String, String> arrayToMap(String[] input) {
    +    if (input == null || input.length == 0) {
    +      return new HashMap<String, String>();
         }
    -    HashMap hm=new HashMap();
    -    for (int i=0;i<input.length;i++){
    -      if (!"".equals(input[i].trim())){
    -        hm.put(input[i],input[i]);
    +    HashMap<String, String> hm = new HashMap<String, String>();
    --- End diff --
    
    Can just use diamond operator on RHS


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message