cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Radzin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-8390) The process cannot access the file because it is being used by another process
Date Tue, 23 Dec 2014 16:42:13 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257183#comment-14257183
] 

Alexander Radzin edited comment on CASSANDRA-8390 at 12/23/14 4:41 PM:
-----------------------------------------------------------------------

Attached Log files (NoHostAvailable.zip) from run that reproduces NoHostAvailable problem.
The {{cqlSync}} successfully finished 11 iterations and failed while doing the iterration
#12. 

This is the client side exception:

{noformat}
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query
failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.TransportException: [localhost/127.0.0.1:9042]
Connection has been closed))
	at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:65)
	at com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:259)
	at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:175)
	at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)
	at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:36)
	at com.clarisite.clingine.dataaccesslayer.cassandra.CQLTest.cqlSync(CQLTest.java:32)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at com.intellij.junit4.JUnit4TestRunnerUtil$IgnoreIgnoredTestJUnit4ClassRunner.runChild(JUnit4TestRunnerUtil.java:269)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at com.intellij.rt.execution.CommandLineWrapper.main(CommandLineWrapper.java:121)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried
for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.TransportException:
[localhost/127.0.0.1:9042] Connection has been closed))
	at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:102)
	at com.datastax.driver.core.RequestHandler$1.run(RequestHandler.java:176)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
{noformat}

Here is the server side exception:

{noformat}
INFO  16:28:12 Compacted 4 sstables to [c:\progs\apache-cassandra-2.1.2\data\data\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-881,].
 297,142 bytes to 295,369 (~99% of original) in 375ms = 0.751162MB/s.  16 total partitions
merged to 13.  Partition merge counts were {1:12, 4:1, }
ERROR 16:28:12 Unable to delete c:\progs\apache-cassandra-2.1.2\data\data\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-879-Data.db
(it will be removed on server restart; we'll also retry after GC)
ERROR 16:28:12 Exception in thread Thread[NonPeriodicTasks:1,5,main]
org.apache.cassandra.io.FSWriteError: java.nio.file.FileSystemException: c:\progs\apache-cassandra-2.1.2\data\data\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-877-Index.db:
The process cannot access the file because it is being used by another process.

        at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:135) ~[apache-cassandra-2.1.2.jar:2.1.2]
        at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:121) ~[apache-cassandra-2.1.2.jar:2.1.2]
        at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:113) ~[apache-cassandra-2.1.2.jar:2.1.2]
        at org.apache.cassandra.io.sstable.SSTableDeletingTask.run(SSTableDeletingTask.java:94)
~[apache-cassandra-2.1.2.jar:2.1.2]
        at org.apache.cassandra.io.sstable.SSTableReader$6.run(SSTableReader.java:665) ~[apache-cassandra-2.1.2.jar:2.1.2]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_45]
        at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_45]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
~[na:1.7.0_45]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
~[na:1.7.0_45]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
        at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: java.nio.file.FileSystemException: c:\progs\apache-cassandra-2.1.2\data\data\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-877-Index.db:
The process cannot access the file because it is being used by another process.

        at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) ~[na:1.7.0_45]
        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) ~[na:1.7.0_45]
        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) ~[na:1.7.0_45]
        at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
~[na:1.7.0_45]
        at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
~[na:1.7.0_45]
        at java.nio.file.Files.delete(Files.java:1077) ~[na:1.7.0_45]
        at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:131) ~[apache-cassandra-2.1.2.jar:2.1.2]
        ... 11 common frames omitted
ERROR 16:28:12 Stopping gossiper
WARN  16:28:12 Stopping gossip by operator request
INFO  16:28:12 Announcing shutdown
INFO  16:28:12 Compacted 4 sstables to [c:\progs\apache-cassandra-2.1.2\data\data\system\schema_columns-296e9c049bec3085827dc17d3df2122a\system-schema_columns-ka-881,].
 302,356 bytes to 301,730 (~99% of original) in 396ms = 0.726647MB/s.  16 total partitions
merged to 13.  Partition merge counts were {1:12, 4:1, }{noformat}


was (Author: alexander_radzin):
Log files from run that reproduces NoHostAvailable problem. The {{cqlSync}} successfully finished
11 iterations and failed while doing the iterration #12. 

This is the client side exception:

{noformat}
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query
failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.TransportException: [localhost/127.0.0.1:9042]
Connection has been closed))
	at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:65)
	at com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:259)
	at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:175)
	at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)
	at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:36)
	at com.clarisite.clingine.dataaccesslayer.cassandra.CQLTest.cqlSync(CQLTest.java:32)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at com.intellij.junit4.JUnit4TestRunnerUtil$IgnoreIgnoredTestJUnit4ClassRunner.runChild(JUnit4TestRunnerUtil.java:269)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at com.intellij.rt.execution.CommandLineWrapper.main(CommandLineWrapper.java:121)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried
for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.TransportException:
[localhost/127.0.0.1:9042] Connection has been closed))
	at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:102)
	at com.datastax.driver.core.RequestHandler$1.run(RequestHandler.java:176)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
{noformat}

Here is the server side exception:

{noformat}
INFO  16:28:12 Compacted 4 sstables to [c:\progs\apache-cassandra-2.1.2\data\data\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-881,].
 297,142 bytes to 295,369 (~99% of original) in 375ms = 0.751162MB/s.  16 total partitions
merged to 13.  Partition merge counts were {1:12, 4:1, }
ERROR 16:28:12 Unable to delete c:\progs\apache-cassandra-2.1.2\data\data\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-879-Data.db
(it will be removed on server restart; we'll also retry after GC)
ERROR 16:28:12 Exception in thread Thread[NonPeriodicTasks:1,5,main]
org.apache.cassandra.io.FSWriteError: java.nio.file.FileSystemException: c:\progs\apache-cassandra-2.1.2\data\data\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-877-Index.db:
The process cannot access the file because it is being used by another process.

        at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:135) ~[apache-cassandra-2.1.2.jar:2.1.2]
        at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:121) ~[apache-cassandra-2.1.2.jar:2.1.2]
        at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:113) ~[apache-cassandra-2.1.2.jar:2.1.2]
        at org.apache.cassandra.io.sstable.SSTableDeletingTask.run(SSTableDeletingTask.java:94)
~[apache-cassandra-2.1.2.jar:2.1.2]
        at org.apache.cassandra.io.sstable.SSTableReader$6.run(SSTableReader.java:665) ~[apache-cassandra-2.1.2.jar:2.1.2]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_45]
        at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_45]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
~[na:1.7.0_45]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
~[na:1.7.0_45]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
        at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: java.nio.file.FileSystemException: c:\progs\apache-cassandra-2.1.2\data\data\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-877-Index.db:
The process cannot access the file because it is being used by another process.

        at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) ~[na:1.7.0_45]
        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) ~[na:1.7.0_45]
        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) ~[na:1.7.0_45]
        at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
~[na:1.7.0_45]
        at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
~[na:1.7.0_45]
        at java.nio.file.Files.delete(Files.java:1077) ~[na:1.7.0_45]
        at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:131) ~[apache-cassandra-2.1.2.jar:2.1.2]
        ... 11 common frames omitted
ERROR 16:28:12 Stopping gossiper
WARN  16:28:12 Stopping gossip by operator request
INFO  16:28:12 Announcing shutdown
INFO  16:28:12 Compacted 4 sstables to [c:\progs\apache-cassandra-2.1.2\data\data\system\schema_columns-296e9c049bec3085827dc17d3df2122a\system-schema_columns-ka-881,].
 302,356 bytes to 301,730 (~99% of original) in 396ms = 0.726647MB/s.  16 total partitions
merged to 13.  Partition merge counts were {1:12, 4:1, }{noformat}

> The process cannot access the file because it is being used by another process
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-8390
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8390
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Ilya Komolkin
>            Assignee: Joshua McKenzie
>             Fix For: 2.1.3
>
>         Attachments: NoHostAvailableLogs.zip
>
>
> 21:46:27.810 [NonPeriodicTasks:1] ERROR o.a.c.service.CassandraDaemon - Exception in
thread Thread[NonPeriodicTasks:1,5,main]
> org.apache.cassandra.io.FSWriteError: java.nio.file.FileSystemException: E:\Upsource_12391\data\cassandra\data\kernel\filechangehistory_t-a277b560764611e48c8e4915424c75fe\kernel-filechangehistory_t-ka-33-Index.db:
The process cannot access the file because it is being used by another process.
>  
>                 at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:135)
~[cassandra-all-2.1.1.jar:2.1.1]
>                 at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:121)
~[cassandra-all-2.1.1.jar:2.1.1]
>                 at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:113) ~[cassandra-all-2.1.1.jar:2.1.1]
>                 at org.apache.cassandra.io.sstable.SSTableDeletingTask.run(SSTableDeletingTask.java:94)
~[cassandra-all-2.1.1.jar:2.1.1]
>                 at org.apache.cassandra.io.sstable.SSTableReader$6.run(SSTableReader.java:664)
~[cassandra-all-2.1.1.jar:2.1.1]
>                 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_71]
>                 at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_71]
>                 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
~[na:1.7.0_71]
>                 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
~[na:1.7.0_71]
>                 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_71]
>                 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_71]
>                 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> Caused by: java.nio.file.FileSystemException: E:\Upsource_12391\data\cassandra\data\kernel\filechangehistory_t-a277b560764611e48c8e4915424c75fe\kernel-filechangehistory_t-ka-33-Index.db:
The process cannot access the file because it is being used by another process.
>  
>                 at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
~[na:1.7.0_71]
>                 at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
~[na:1.7.0_71]
>                 at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
~[na:1.7.0_71]
>                 at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
~[na:1.7.0_71]
>                 at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
~[na:1.7.0_71]
>                 at java.nio.file.Files.delete(Files.java:1079) ~[na:1.7.0_71]
>                 at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:131)
~[cassandra-all-2.1.1.jar:2.1.1]
>                 ... 11 common frames omitted



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message