cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Radzin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-8390) The process cannot access the file because it is being used by another process
Date Wed, 10 Dec 2014 09:27:12 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240844#comment-14240844
] 

Alexander Radzin edited comment on CASSANDRA-8390 at 12/10/14 9:26 AM:
-----------------------------------------------------------------------

I have the same issue with Windows 8. Here is the "DiskAccessMode" line that I found in system.log
of cassandra:

{noformat}
INFO  [main] 2014-12-09 16:07:25,985 DatabaseDescriptor.java:203 - DiskAccessMode 'auto' determined
to be mmap, indexAccessMode is mmap
{noformat}

I have several lines like this. 

Important: when this happens client gets {{NoHostAvailableException}} and stops working that
requires restart of cassandra.

{noformat}
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query
failed (no host was tried)
	at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:65)
	at com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:259)
	at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:175)
	at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)
	at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:36)
	at com.clarisite.clingine.dataaccesslayer.cassandra.CQLTest1.cqlSync(CQLTest1.java:56)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at com.intellij.rt.execution.CommandLineWrapper.main(CommandLineWrapper.java:121)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried
for query failed (no host was tried)
	at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:102)
	at com.datastax.driver.core.SessionManager.execute(SessionManager.java:461)
	at com.datastax.driver.core.SessionManager.executeQuery(SessionManager.java:497)
	at com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:87)
	... 34 more
{noformat}


Here are the conditions that make this problem to be reproduced. 
Our application creates new keyspace every day. The keyspace contains about 60 tables. The
issue happened on production relatively seldom, however it happens in testing environment
all the time because each test case creates keyspace again. I guess that the problem is not
specifically in creating keyspace and tables because sometimes the problem happens when trying
to run {{truncate}}. 

Cassandra DB is running using default settings. The client code looks like the following:

{noformat}
		Cluster cluster = Cluster.builder().addContactPoint("localhost").build();
		Session session = cluster.connect();
		String year = "2013";
		for (int i = 1; i <= 12; i++) {
			String yearMonth = year + i;
			for (String template : cql.split("\\n")) {
				String query = String.format(template, yearMonth);
				System.out.println(query);
				session.execute(query);
			}
		}
{noformat}
Where {{cql}} contains  {{create keyspace}} and a lot of {{create table}} statements. 

Interesting fact is that problem _does not appear_ when using asynchronous call:

{noformat}
		Collection<ResultSetFuture> futures = new ArrayList<>();

		Cluster cluster = Cluster.builder().addContactPoint("localhost").build();
		Session session = cluster.connect();
		String year = "2013";
		for (int i = 1; i <= 1200; i++) {
			String yearMonth = year + i;
			for (String template : cql.split("\\n")) {
				String query = String.format(template, yearMonth);
				System.out.println(query);
				ResultSetFuture future = session.executeAsync(query);
				futures.add(future);
			}
		}

		Futures.successfulAsList(futures);
{noformat}

Although this can be a temporary workaround I will try to use the problem itself is IMHO extremely
critical. 

Full source code can be found [here|https://gist.github.com/alexradzin/9223fc16e95318e017ec].



was (Author: alexander_radzin):
I have the same issue with Windows 8. Here is the "DiskAccessMode" line that I found in system.log
of cassandra:

{noformat}
INFO  [main] 2014-12-09 16:07:25,985 DatabaseDescriptor.java:203 - DiskAccessMode 'auto' determined
to be mmap, indexAccessMode is mmap
{noformat}

I have several lines like this. 

Here are the conditions that make this problem to be reproduced. 
Our application creates new keyspace every day. The keyspace contains about 60 tables. The
issue happened on production relatively seldom, however it happens in testing environment
all the time because each test case creates keyspace again. I guess that the problem is not
specifically in creating keyspace and tables because sometimes the problem happens when trying
to run {{truncate}}. 

Cassandra DB is running using default settings. The client code looks like the following:

{noformat}
		Cluster cluster = Cluster.builder().addContactPoint("localhost").build();
		Session session = cluster.connect();
		String year = "2013";
		for (int i = 1; i <= 12; i++) {
			String yearMonth = year + i;
			for (String template : cql.split("\\n")) {
				String query = String.format(template, yearMonth);
				System.out.println(query);
				session.execute(query);
			}
		}
{noformat}
Where {{cql}} contains  {{create keyspace}} and a lot of {{create table}} statements. 

Interesting fact is that problem _does not appear_ when using asynchronous call:

{noformat}
		Collection<ResultSetFuture> futures = new ArrayList<>();

		Cluster cluster = Cluster.builder().addContactPoint("localhost").build();
		Session session = cluster.connect();
		String year = "2013";
		for (int i = 1; i <= 1200; i++) {
			String yearMonth = year + i;
			for (String template : cql.split("\\n")) {
				String query = String.format(template, yearMonth);
				System.out.println(query);
				ResultSetFuture future = session.executeAsync(query);
				futures.add(future);
			}
		}

		Futures.successfulAsList(futures);
{noformat}

Although this can be a temporary workaround I will try to use the problem itself is IMHO extremely
critical. 

Full source code can be found [here|https://gist.github.com/alexradzin/9223fc16e95318e017ec].


> The process cannot access the file because it is being used by another process
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-8390
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8390
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Ilya Komolkin
>            Assignee: Joshua McKenzie
>             Fix For: 2.1.3
>
>
> 21:46:27.810 [NonPeriodicTasks:1] ERROR o.a.c.service.CassandraDaemon - Exception in
thread Thread[NonPeriodicTasks:1,5,main]
> org.apache.cassandra.io.FSWriteError: java.nio.file.FileSystemException: E:\Upsource_12391\data\cassandra\data\kernel\filechangehistory_t-a277b560764611e48c8e4915424c75fe\kernel-filechangehistory_t-ka-33-Index.db:
The process cannot access the file because it is being used by another process.
>  
>                 at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:135)
~[cassandra-all-2.1.1.jar:2.1.1]
>                 at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:121)
~[cassandra-all-2.1.1.jar:2.1.1]
>                 at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:113) ~[cassandra-all-2.1.1.jar:2.1.1]
>                 at org.apache.cassandra.io.sstable.SSTableDeletingTask.run(SSTableDeletingTask.java:94)
~[cassandra-all-2.1.1.jar:2.1.1]
>                 at org.apache.cassandra.io.sstable.SSTableReader$6.run(SSTableReader.java:664)
~[cassandra-all-2.1.1.jar:2.1.1]
>                 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_71]
>                 at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_71]
>                 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
~[na:1.7.0_71]
>                 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
~[na:1.7.0_71]
>                 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_71]
>                 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_71]
>                 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> Caused by: java.nio.file.FileSystemException: E:\Upsource_12391\data\cassandra\data\kernel\filechangehistory_t-a277b560764611e48c8e4915424c75fe\kernel-filechangehistory_t-ka-33-Index.db:
The process cannot access the file because it is being used by another process.
>  
>                 at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
~[na:1.7.0_71]
>                 at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
~[na:1.7.0_71]
>                 at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
~[na:1.7.0_71]
>                 at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
~[na:1.7.0_71]
>                 at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
~[na:1.7.0_71]
>                 at java.nio.file.Files.delete(Files.java:1079) ~[na:1.7.0_71]
>                 at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:131)
~[cassandra-all-2.1.1.jar:2.1.1]
>                 ... 11 common frames omitted



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message