nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NUTCH-2441) ARG_SEGMENT usage
Date Thu, 18 Jan 2018 19:05:00 GMT

    [ https://issues.apache.org/jira/browse/NUTCH-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330995#comment-16330995
] 

Hudson commented on NUTCH-2441:
-------------------------------

SUCCESS: Integrated in Jenkins build Nutch-trunk #3497 (See [https://builds.apache.org/job/Nutch-trunk/3497/])
fix for NUTCH-2441 ARG_SEGMENT fix for REST API (semyon.semyonov: [https://github.com/apache/nutch/commit/34083703027a3967bd23e047f02b49fe48fab6ba])
* (edit) src/java/org/apache/nutch/metadata/Nutch.java
* (edit) src/java/org/apache/nutch/parse/ParseSegment.java
* (edit) src/java/org/apache/nutch/fetcher/Fetcher.java
* (edit) src/java/org/apache/nutch/indexer/IndexingJob.java
* (edit) src/java/org/apache/nutch/crawl/CrawlDb.java
* (edit) src/java/org/apache/nutch/crawl/LinkDb.java
review fix of formating and comments for NUTCH-2441 ARG_SEGMENT usage (semyon.semyonov: [https://github.com/apache/nutch/commit/631b747826771b60a879ceba2b2773ebc1361a71])
* (edit) src/java/org/apache/nutch/crawl/CrawlDb.java
* (edit) src/java/org/apache/nutch/indexer/IndexingJob.java
* (edit) src/java/org/apache/nutch/parse/ParseSegment.java
* (edit) src/java/org/apache/nutch/fetcher/Fetcher.java
* (edit) src/java/org/apache/nutch/crawl/LinkDb.java
* (edit) src/java/org/apache/nutch/metadata/Nutch.java


> ARG_SEGMENT usage
> -----------------
>
>                 Key: NUTCH-2441
>                 URL: https://issues.apache.org/jira/browse/NUTCH-2441
>             Project: Nutch
>          Issue Type: Improvement
>          Components: metadata
>    Affects Versions: 1.13
>            Reporter: Semyon Semyonov
>            Priority: Major
>             Fix For: 1.15
>
>         Attachments: metadataARG_SEGMENT.patch
>
>
> The class metadata/Nutch.java  public static final String ARG_SEGMENT = "segment" is
not used correctly. In some cases Fetcher and ParseSegment it is interpreted as a single segmenet,
in others CrawlDb, LinkDb, IndexingJob as an array of segments. Such misunderstanding leads
to inconsistency of usage of the parameter.
> After a discussion with [~wastl-nagel]  the proposed solution is to allow the usage of
both array and a string in all cases. That gives an opportunity to not introduce the broken
changes.
> A path is proposed.
>  *The question left is refactoring, all these five components share the same code(two
versions of the same code to be precise). Shouldn't we extract a method and reduce duplicates?
 *



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message