systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias Boehm" <mbo...@us.ibm.com>
Subject Re: machine learning - Some tests failure when build systemML project -StackOverflow
Date Tue, 12 Apr 2016 06:34:14 GMT

ok great, so it's indeed an issue of file system permissions ("[...] is not
publicly accessable [...]"). Could you please recursively set 755
permissions for everything under systemml (temporarily) just to see if this
fixes the issue. Thanks.

Regards,
Matthias



From:	281165273@qq.com
To:	"dev" <dev@systemml.incubator.apache.org>
Date:	04/11/2016 11:26 PM
Subject:	Re:   machine learning - Some tests failure when build systemML
            project -StackOverflow



Stacktrace for
org.apache.sysml.test.integration.functions.data.FullReblockTest, I just
run command 'mvn test -Dtest=FullReblockTest -e'.


org.apache.sysml.runtime.DMLRuntimeException:
org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error in
program block generated from statement block between lines 22 and 25 --
Error evaluating instruction: jobtype = CSV_REBLOCK
input labels = [pREADA1, pREADA2]
recReader inst =
rand inst =
mapper inst =
shuffle inst = MR°csvrblk°0·MATRIX·DOUBLE°2·MATRIX·DOUBLE°1000°1000°
false°,°true°0.0‡MR°csvrblk°1·MATRIX·DOUBLE°3·MATRIX·DOUBLE°1000°1000°
false°,°true°0.0
agg inst =
other inst =
output labels = [pVar89, pVar90]
result indices = ,2,3
num reducers = 10
replication = 1

    at org.apache.sysml.runtime.controlprogram.Program.execute
(Program.java:152)
    at org.apache.sysml.api.DMLScript.execute(DMLScript.java:677)
    at org.apache.sysml.api.DMLScript.executeScript(DMLScript.java:335)
    at org.apache.sysml.api.DMLScript.main(DMLScript.java:190)
    at org.apache.sysml.test.integration.AutomatedTestBase.runTest
(AutomatedTestBase.java:1171)
    at
org.apache.sysml.test.integration.functions.data.FullReblockTest.runReblockTest
(FullReblockTest.java:459)
    at
org.apache.sysml.test.integration.functions.data.FullReblockTest.testCSVMultipleMSparseMR
(FullReblockTest.java:403)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall
(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run
(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively
(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate
(InvokeMethod.java:17)
    at org.junit.internal.runners.statements.RunBefores.evaluate
(RunBefores.java:26)
    at org.junit.internal.runners.statements.RunAfters.evaluate
(RunAfters.java:27)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild
(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild
(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute
(JUnit4Provider.java:283)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun
(JUnit4Provider.java:173)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet
(JUnit4Provider.java:153)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke
(JUnit4Provider.java:128)
    at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader
(ForkedBooter.java:203)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess
(ForkedBooter.java:155)
    at org.apache.maven.surefire.booter.ForkedBooter.main
(ForkedBooter.java:103)
Caused by: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime
error in program block generated from statement block between lines 22 and
25 -- Error evaluating instruction: jobtype = CSV_REBLOCK
input labels = [pREADA1, pREADA2]
recReader inst =
rand inst =
mapper inst =
shuffle inst = MR°csvrblk°0·MATRIX·DOUBLE°2·MATRIX·DOUBLE°1000°1000°
false°,°true°0.0‡MR°csvrblk°1·MATRIX·DOUBLE°3·MATRIX·DOUBLE°1000°1000°
false°,°true°0.0
agg inst =
other inst =
output labels = [pVar89, pVar90]
result indices = ,2,3
num reducers = 10
replication = 1

    at
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction
(ProgramBlock.java:333)
    at
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeInstructions
(ProgramBlock.java:222)
    at org.apache.sysml.runtime.controlprogram.ProgramBlock.execute
(ProgramBlock.java:166)
    at org.apache.sysml.runtime.controlprogram.Program.execute
(Program.java:145)
    ... 32 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException:
java.io.IOException: java.util.concurrent.ExecutionException:
java.io.IOException: Resource
file:/home/grapebaba/Documents/projects/incubator-systemml/target/testTemp/functions/data/FullReblockTest/MultipleReblockTest/scratch_space/_p20807_127.0.0.1/TmpOutput28/part-00000
 is not publicly accessable and as such cannot be part of the public cache.
    at org.apache.sysml.lops.runtime.RunMRJobs.submitJob
(RunMRJobs.java:334)
    at org.apache.sysml.lops.runtime.RunMRJobs.prepareAndSubmitJob
(RunMRJobs.java:142)
    at
org.apache.sysml.runtime.instructions.MRJobInstruction.processInstruction
(MRJobInstruction.java:1499)
    at
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction
(ProgramBlock.java:303)
    ... 35 more
Caused by: java.io.IOException: java.util.concurrent.ExecutionException:
java.io.IOException: Resource
file:/home/grapebaba/Documents/projects/incubator-systemml/target/testTemp/functions/data/FullReblockTest/MultipleReblockTest/scratch_space/_p20807_127.0.0.1/TmpOutput28/part-00000
 is not publicly accessable and as such cannot be part of the public cache.
    at org.apache.hadoop.mapred.LocalDistributedCacheManager.setup
(LocalDistributedCacheManager.java:149)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>
(LocalJobRunner.java:163)
    at org.apache.hadoop.mapred.LocalJobRunner.submitJob
(LocalJobRunner.java:731)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal
(JobSubmitter.java:432)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs
(UserGroupInformation.java:1556)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
    at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
    at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs
(UserGroupInformation.java:1556)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal
(JobClient.java:557)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:833)
    at org.apache.sysml.runtime.matrix.CSVReblockMR.runCSVReblockJob
(CSVReblockMR.java:468)
    at org.apache.sysml.runtime.matrix.CSVReblockMR.runJob
(CSVReblockMR.java:280)
    at org.apache.sysml.lops.runtime.RunMRJobs.submitJob
(RunMRJobs.java:256)
    ... 38 more
Caused by: java.util.concurrent.ExecutionException: java.io.IOException:
Resource
file:/home/grapebaba/Documents/projects/incubator-systemml/target/testTemp/functions/data/FullReblockTest/MultipleReblockTest/scratch_space/_p20807_127.0.0.1/TmpOutput28/part-00000
 is not publicly accessable and as such cannot be part of the public cache.
    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.util.concurrent.FutureTask.get(FutureTask.java:192)
    at org.apache.hadoop.mapred.LocalDistributedCacheManager.setup
(LocalDistributedCacheManager.java:145)
    ... 58 more
Caused by: java.io.IOException: Resource
file:/home/grapebaba/Documents/projects/incubator-systemml/target/testTemp/functions/data/FullReblockTest/MultipleReblockTest/scratch_space/_p20807_127.0.0.1/TmpOutput28/part-00000
 is not publicly accessable and as such cannot be part of the public cache.
    at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:257)
    at org.apache.hadoop.yarn.util.FSDownload.access$000
(FSDownload.java:60)
    at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:355)
    at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:353)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs
(UserGroupInformation.java:1556)
    at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:352)
    at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:59)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker
(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)





------------------ Original ------------------
From:  "Matthias Boehm";<mboehm@us.ibm.com>;
Date:  Tue, Apr 12, 2016 01:18 PM
To:  "dev"<dev@systemml.incubator.apache.org>;
Cc:  "葡萄??"<281165273@qq.com>;
Subject:  Re:  machine learning - Some tests failure when build systemML
project -StackOverflow




could you please pick one failing test to start with and provide the
respective full stacktrace (potentially rerun it individually w/
maven/junit)? This would already reveal common setup issues like file
permissions, missing directories, etc. Thanks.

Regards,
Matthias

281165273---04/11/2016 10:11:21 PM---Thanks your quick response. But I saw
many different errors occur on different test case. I archived

From:        281165273@qq.com
To:        "dev" <dev@systemml.incubator.apache.org>
Date:        04/11/2016 10:11 PM
Subject:        Re:  machine learning - Some tests failure when build
systemML project -Stack Overflow





Thanks your quick response. But I saw many different errors occur on
different test case. I archived the failsafe-report. Would you like help to
see it?


------------------ Original ------------------
From:  "Matthias Boehm";<mboehm@us.ibm.com>;
Date:  Tue, Apr 12, 2016 11:31 AM
To:  "dev"<dev@systemml.incubator.apache.org>;
Cc:  "葡萄??"<281165273@qq.com>;
Subject:  Re: machine learning - Some tests failure when build systemML
project -Stack Overflow

well the error is not coming from R but from SystemML's runtime. Could you
please provide the full stacktrace to see what is going on here?

Regards,
Matthias

281165273---04/11/2016 08:22:18 PM---Sorry to bother you guys, I am a
developer from IBM and interesting in this project. I just ask a qu

From: 281165273@qq.com
To: "dev" <dev@systemml.incubator.apache.org>
Date: 04/11/2016 08:22 PM
Subject: machine learning - Some tests failure when build systemML project
- Stack Overflow






Sorry to bother you guys, I am a developer from IBM and interesting in this
project. I just ask a question (
http://stackoverflow.com/questions/36562951/some-tests-failure-when-build-systemml-project
) on stackoverflow, but I don't find any tag for this project, could I post
it in this mail list?

Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message