nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stas Batururimi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NUTCH-2676) Update to the latest selenium and add code to use chrome and firefox headless mode with the remote web driver
Date Mon, 19 Nov 2018 07:33:00 GMT

    [ https://issues.apache.org/jira/browse/NUTCH-2676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691338#comment-16691338
] 

Stas Batururimi commented on NUTCH-2676:
----------------------------------------

 
Still getting the same error.
The ivy version is 2.4.0.
I'm building the project by replicating the environment with a docker container. That is:

{code:java}
docker run -dit -v $PWD:/root/nutch -w /root --name nutch_container ubuntu:16.04 bash

then installing all dependencies
apt upgrade
 apt update
 apt install -y ant openssh-server vim telnet git rsync curl openjdk-8-jdk-headless
 echo 'export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64' >> $HOME/.bashrc
 source ~/.bashrc
{code}

 And compiling the project with

{code:java}
 ant runtime
{code}

 or as specified above:


{code:java}
 run ant -d clean runtime
{code}


 The result I always got are:

{code:java}
 [ivy:resolve] report for org.apache.nutch#parse-tika;working@8ea54697ddb4 default produced
in /root/.ivy2/cache/org.apache.nutch-parse-tika-default.xml
 [ivy:resolve] resolve done (47605ms resolve - 16768ms download)
 [ivy:resolve] 
 [ivy:resolve] :: problems summary ::
 [ivy:resolve] :::: WARNINGS
 [ivy:resolve] [FAILED ] javax.ws.rs#javax.ws.rs-api;2.1!javax.ws.rs-api.${packaging.type}:
(0ms)
 [ivy:resolve] ==== local: tried
 [ivy:resolve] /root/.ivy2/local/javax.ws.rs/javax.ws.rs-api/2.1/${packaging.type}s/javax.ws.rs-api.${packaging.type}
 [ivy:resolve] ==== maven2: tried
 [ivy:resolve] http://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.${packaging.type}
 [ivy:resolve] ==== apache-snapshot: tried
 [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.${packaging.type}
 [ivy:resolve] ==== sonatype: tried
 [ivy:resolve] http://oss.sonatype.org/content/repositories/releases/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.${packaging.type}
 [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
 [ivy:resolve] :: FAILED DOWNLOADS ::
 [ivy:resolve] :: ^ see resolution messages for details ^ ::
 [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
 [ivy:resolve] :: javax.ws.rs#javax.ws.rs-api;2.1!javax.ws.rs-api.${packaging.type}
 [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
 [ivy:resolve] 
 [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
 [ant] Exiting /root/nutch/src/plugin/parse-tika/build.xml.
 [ant] Exiting /root/nutch/src/plugin/build.xml.
BUILD FAILED
 /root/nutch/build.xml:116: The following error occurred while executing this line:
 /root/nutch/src/plugin/build.xml:68: The following error occurred while executing this line:
 /root/nutch/src/plugin/build-plugin.xml:229: impossible to resolve dependencies:
 resolve failed - see output for details
 at org.apache.ivy.ant.IvyResolve.doExecute(IvyResolve.java:337)
 at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:271)
 at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293)
 at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
 at org.apache.tools.ant.Task.perform(Task.java:348)
 at org.apache.tools.ant.Target.execute(Target.java:435)
 at org.apache.tools.ant.Target.performTasks(Target.java:456)
 at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405)
 at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
 at org.apache.tools.ant.Project.executeTargets(Project.java:1260)
 at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:441)
 at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293)
 at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
 at org.apache.tools.ant.Task.perform(Task.java:348)
 at org.apache.tools.ant.Target.execute(Target.java:435)
 at org.apache.tools.ant.Target.performTasks(Target.java:456)
 at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405)
 at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
 at org.apache.tools.ant.Project.executeTargets(Project.java:1260)
 at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:441)
 at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293)
 at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
 at org.apache.tools.ant.Task.perform(Task.java:348)
 at org.apache.tools.ant.Target.execute(Target.java:435)
 at org.apache.tools.ant.Target.performTasks(Target.java:456)
 at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405)
 at org.apache.tools.ant.Project.executeTarget(Project.java:1376)
 at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
 at org.apache.tools.ant.Project.executeTargets(Project.java:1260)
 at org.apache.tools.ant.Main.runBuild(Main.java:853)
 at org.apache.tools.ant.Main.startAnt(Main.java:235)
 at org.apache.tools.ant.launch.Launcher.run(Launcher.java:285)
 at org.apache.tools.ant.launch.Launcher.main(Launcher.java:112)
Total time: 7 minutes 39 seconds
{code}


 

> Update to the latest selenium and add code to use chrome and firefox headless mode with
the remote web driver
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: NUTCH-2676
>                 URL: https://issues.apache.org/jira/browse/NUTCH-2676
>             Project: Nutch
>          Issue Type: New Feature
>          Components: protocol
>    Affects Versions: 1.15
>            Reporter: Stas Batururimi
>            Priority: Major
>             Fix For: 1.16
>
>         Attachments: Screenshot 2018-11-16 at 18.15.52.png
>
>
> * Selenium needs to be updated
>  * missing remote web driver for chrome 
>  * necessity to add headless mode for both remote WebDriverBase Firefox & Chrome
>  * use case with Selenium grid using docker (1 hub docker container, several nodes in
different docker containers, Nutch in another docker container, streaming to Apache Solr in
docker container, that is at least 4 different docker containers)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message