lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dawid Weiss (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-5914) Almost all Solr tests no longer cleanup their temp dirs on Windows
Date Sat, 29 Mar 2014 21:20:22 GMT

    [ https://issues.apache.org/jira/browse/SOLR-5914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13954447#comment-13954447
] 

Dawid Weiss commented on SOLR-5914:
-----------------------------------

I looked at the current state of the branch, Mark, and I strongly disagree with having this
in TestUtil:
{code}
  public static File createTempDir(String name, File tmpDir, boolean ensureCleanedUp)
{code}

It's creating a gateway for bad code to proliferate. A temporary folder WILL be removed after
a suite is done, there should be no way to "opt out" from this check. I understand sometimes
it's useful to keep temp. files and it should be possible, but NOT AT THE CODE LEVEL (it should
be a result of an explicit action of the developer).

My suggestion is to change SolrTestCaseJ4 and instead of putting the logic to keep temporary
folders in there I'd rather:

- declare a global sys property that would prevent removing temporary folders (placed in TestUtil;
but we can keep solr.test.leavetmpdir as an alias). In fact there already is a similar property
for the runner itself:
{code}
# Don't remove temporary files under slave directories, even if
# the test passes.
ant -Dtests.leaveTemporary=true
{code}
so we could just adopt it here too.

- make an annotation called DirtyHarry (suggestions welcome) to mark suites which are known
offenders of the default behavior. Classes annotated with DirtyHarry wouldn't fail if they
leave undeletable garbage behind (but would print a warning).

I also don't quite understand why you insist on things like this:
{code}
 rootTmpDir = TestUtil.createTempDir("solrtest-" + cname, null, ensureClosed);
 initCoreDataDir = TestUtil.createTempDir("solrtest-" + cname, rootTmpDir, ensureClosed);
{code}

Why 'solrtest-*' prefix? Isn't class name enough? And why create a temporary folder under
a temporary folder (if we know rootTmpDir will always be empty, even on a run when previous
runs left their temporary folders a new, empty folder uniquely suffixed will be created.

As for this:
{code}
          // TODO: tmp files should already get cleaned up by the test framework, but
          // we still do it here as well, so that we clean up as much as we can, even
          // when a test is the SuppressTempDirCleanUp annotation
{code}
it isn't true. The rule to clean up temporary files is fired almost at the very end of processing,
after all the after class rules have already been executed. The reason for this is that you
don't want to clean up any temporary files for failed tests/ suites and you know this only
after everything else has been executed.

Let me know what you think, I'll make the above changes and make them ready for your review.

> Almost all Solr tests no longer cleanup their temp dirs on Windows
> ------------------------------------------------------------------
>
>                 Key: SOLR-5914
>                 URL: https://issues.apache.org/jira/browse/SOLR-5914
>             Project: Solr
>          Issue Type: Bug
>          Components: Tests
>    Affects Versions: 4.8
>            Reporter: Uwe Schindler
>            Assignee: Dawid Weiss
>            Priority: Critical
>             Fix For: 4.8
>
>         Attachments: SOLR-5914 .patch, SOLR-5914 .patch, branch4x-jenkins.png, build-plugin.jpg,
trunk-jenkins.png
>
>
> Recently the Windows Jenkins Build server has the problem of all-the time running out
of disk space. This machine runs 2 workspaces (4.x and trunk) and has initially 8 Gigabytes
of free SSD disk space.
> Because of the recently all-the time failing tests, the test framework does not forcefully
clean up the "J0" working folders after running tests. This leads to the fact, that the workspace
is filled with tons of Solr Home dirs. I tried this on my local machine:
> - run ant test
> - go to build/.../test/J0 and watch folders appearing: Almost every test no longer cleans
up after shutting down, leaving a million of files there. This is approx 3 to 4 Gigabytes!!!
> In Lucene the folders are correctly removed. This has happened recently, so i think we
have some code like ([~erickerickson] !!!):
> {{new Properties().load(new FileInputStream(...))}} that does not close the files. Because
of this, the test's afterClass cannot clean up folders anymore. If you look in the test log,
you see messages like "{{!!!! WARNING: best effort to remove C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.TestShortCircuitedRequests-1395693845226
FAILED !!!!!}}" all the time.
> So if anybody committed some changes that might not close files correctly, please fix!
Otherwise I have to disable testing on windows - and I will no longer run solr, tests, too:
My local computer also uses gigabytes of temp space after running tests!



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message