lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Miller (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-5914) Almost all Solr tests no longer cleanup their temp dirs on Windows
Date Thu, 27 Mar 2014 22:08:15 GMT

    [ https://issues.apache.org/jira/browse/SOLR-5914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950011#comment-13950011
] 

Mark Miller commented on SOLR-5914:
-----------------------------------

bq. You see it depends on what you consider "wrong".

Let's also come to agreement here. SolrjTestCase4j has a temporary directory that it tries
to clean after the test and that you can set a special system property to prevent cleaning.
This is obviously where Solr tests should be creating their temporary files - SolrjTestCase4j
is the root of all Solr tests (or is intended to be). So clearly, this working haphazardly
is wrong.

I've been part of that happening - LuceneTestCase#TEMP_DIR was just too tempting - I remember
it's allure and using it. It got to the point where a ton of stuff was no longer under the
cleaned up tmp dir and we got into this situation. It was not always like this - we had much,
much less leakage.

So I have fixed that. I didn't fix that the dataDir is conflated hopelessly between two worlds,
I didn't fix any sugar api's or code, and I didn't address any other systemic issue. I just
fixed what has clearly been borked and was supposed to work for eons now. If you set the right
solr sys prop, you dont less any tmp files. When the test is over, it cleans up. That is back
to working.

If we can agree on that, perhaps we can start on the path of improvement. I just started integrating
into your branch. Let's see what I come up with and push forward together from there.

> Almost all Solr tests no longer cleanup their temp dirs on Windows
> ------------------------------------------------------------------
>
>                 Key: SOLR-5914
>                 URL: https://issues.apache.org/jira/browse/SOLR-5914
>             Project: Solr
>          Issue Type: Bug
>          Components: Tests
>    Affects Versions: 4.8
>            Reporter: Uwe Schindler
>            Assignee: Dawid Weiss
>            Priority: Critical
>             Fix For: 4.8
>
>         Attachments: SOLR-5914 .patch, SOLR-5914 .patch, branch4x-jenkins.png, build-plugin.jpg,
trunk-jenkins.png
>
>
> Recently the Windows Jenkins Build server has the problem of all-the time running out
of disk space. This machine runs 2 workspaces (4.x and trunk) and has initially 8 Gigabytes
of free SSD disk space.
> Because of the recently all-the time failing tests, the test framework does not forcefully
clean up the "J0" working folders after running tests. This leads to the fact, that the workspace
is filled with tons of Solr Home dirs. I tried this on my local machine:
> - run ant test
> - go to build/.../test/J0 and watch folders appearing: Almost every test no longer cleans
up after shutting down, leaving a million of files there. This is approx 3 to 4 Gigabytes!!!
> In Lucene the folders are correctly removed. This has happened recently, so i think we
have some code like ([~erickerickson] !!!):
> {{new Properties().load(new FileInputStream(...))}} that does not close the files. Because
of this, the test's afterClass cannot clean up folders anymore. If you look in the test log,
you see messages like "{{!!!! WARNING: best effort to remove C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.TestShortCircuitedRequests-1395693845226
FAILED !!!!!}}" all the time.
> So if anybody committed some changes that might not close files correctly, please fix!
Otherwise I have to disable testing on windows - and I will no longer run solr, tests, too:
My local computer also uses gigabytes of temp space after running tests!



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message