lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Jelsma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-5714) You should be able to use one pool of memory for multiple collection's HDFS block caches.
Date Tue, 04 Mar 2014 13:46:26 GMT

    [ https://issues.apache.org/jira/browse/SOLR-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13919402#comment-13919402
] 

Markus Jelsma commented on SOLR-5714:
-------------------------------------

Hi Mark! I get stuff like:
{code}
common.compile-test:
    [javac] Compiling 365 source files to /home/markus/src/solr/trunk/solr/build/solr-core/classes/test
    [javac] /home/markus/src/solr/trunk/solr/core/src/test/org/apache/solr/cloud/hdfs/HdfsWriteToMultipleCollectionsTest.java:35:
error: cannot find symbol
    [javac] import org.apache.solr.cloud.StopableIndexingThread;
    [javac]                             ^
    [javac]   symbol:   class StopableIndexingThread
    [javac]   location: package org.apache.solr.cloud
    [javac] /home/markus/src/solr/trunk/solr/core/src/test/org/apache/solr/cloud/ChaosMonkeyNothingIsSafeTest.java:134:
error: no suitable constructor found for StopableIndexingThread(SolrServer,CloudSolrServer,String,boolean)
    [javac]         StopableIndexingThread indexThread = new StopableIndexingThread(controlClient,
cloudClient, Integer.toString(i), true);
    [javac]                                              ^
    [javac]     constructor AbstractFullDistribZkTestBase.StopableIndexingThread.AbstractFullDistribZkTestBase.StopableIndexingThread(String,boolean,int)
is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac]     constructor AbstractFullDistribZkTestBase.StopableIndexingThread.AbstractFullDistribZkTestBase.StopableIndexingThread(String,boolean)
is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac] /home/markus/src/solr/trunk/solr/core/src/test/org/apache/solr/cloud/ChaosMonkeyNothingIsSafeTest.java:273:
error: no suitable constructor found for StopableIndexingThread(SolrServer,CloudSolrServer,String,boolean)
    [javac]       super(controlClient, cloudClient, id, doDeletes);
    [javac]       ^
    [javac]     constructor AbstractFullDistribZkTestBase.StopableIndexingThread.AbstractFullDistribZkTestBase.StopableIndexingThread(String,boolean,int)
is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac]     constructor AbstractFullDistribZkTestBase.StopableIndexingThread.AbstractFullDistribZkTestBase.StopableIndexingThread(String,boolean)
is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac] /home/markus/src/solr/trunk/solr/core/src/test/org/apache/solr/cloud/ChaosMonkeySafeLeaderTest.java:111:
error: no suitable constructor found for StopableIndexingThread(SolrServer,CloudSolrServer,String,boolean)
    [javac]       StopableIndexingThread indexThread = new StopableIndexingThread(controlClient,
cloudClient, Integer.toString(i), true);
    [javac]                                            ^
    [javac]     constructor AbstractFullDistribZkTestBase.StopableIndexingThread.AbstractFullDistribZkTestBase.StopableIndexingThread(String,boolean,int)
is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac]     constructor AbstractFullDistribZkTestBase.StopableIndexingThread.AbstractFullDistribZkTestBase.StopableIndexingThread(String,boolean)
is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac] /home/markus/src/solr/trunk/solr/core/src/test/org/apache/solr/cloud/RecoveryZkTest.java:69:
error: no suitable constructor found for StopableIndexingThread(SolrServer,CloudSolrServer,String,boolean,int)
    [javac]     indexThread = new StopableIndexingThread(controlClient, cloudClient, "1",
true, maxDoc);
    [javac]                   ^
    [javac]     constructor AbstractFullDistribZkTestBase.StopableIndexingThread.AbstractFullDistribZkTestBase.StopableIndexingThread(String,boolean,int)
is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac]     constructor AbstractFullDistribZkTestBase.StopableIndexingThread.AbstractFullDistribZkTestBase.StopableIndexingThread(String,boolean)
is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac] /home/markus/src/solr/trunk/solr/core/src/test/org/apache/solr/cloud/RecoveryZkTest.java:72:
error: no suitable constructor found for StopableIndexingThread(SolrServer,CloudSolrServer,String,boolean,int)
    [javac]     indexThread2 = new StopableIndexingThread(controlClient, cloudClient, "2",
true, maxDoc);
    [javac]                    ^
    [javac]     constructor AbstractFullDistribZkTestBase.StopableIndexingThread.AbstractFullDistribZkTestBase.StopableIndexingThread(String,boolean,int)
is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac]     constructor AbstractFullDistribZkTestBase.StopableIndexingThread.AbstractFullDistribZkTestBase.StopableIndexingThread(String,boolean)
is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac] /home/markus/src/solr/trunk/solr/core/src/test/org/apache/solr/cloud/hdfs/HdfsWriteToMultipleCollectionsTest.java:102:
error: AbstractFullDistribZkTestBase.StopableIndexingThread is not public in AbstractFullDistribZkTestBase;
cannot be accessed from outside package
    [javac]     List<StopableIndexingThread> threads = new ArrayList<StopableIndexingThread>();
    [javac]          ^
    [javac] /home/markus/src/solr/trunk/solr/core/src/test/org/apache/solr/cloud/hdfs/HdfsWriteToMultipleCollectionsTest.java:102:
error: AbstractFullDistribZkTestBase.StopableIndexingThread is not public in AbstractFullDistribZkTestBase;
cannot be accessed from outside package
    [javac]     List<StopableIndexingThread> threads = new ArrayList<StopableIndexingThread>();
    [javac]                                                          ^
    [javac] /home/markus/src/solr/trunk/solr/core/src/test/org/apache/solr/cloud/hdfs/HdfsWriteToMultipleCollectionsTest.java:107:
error: AbstractFullDistribZkTestBase.StopableIndexingThread is not public in AbstractFullDistribZkTestBase;
cannot be accessed from outside package
    [javac]       StopableIndexingThread indexThread = new StopableIndexingThread(null, server,
"1", true, docCount);
    [javac]       ^
    [javac] /home/markus/src/solr/trunk/solr/core/src/test/org/apache/solr/cloud/hdfs/HdfsWriteToMultipleCollectionsTest.java:107:
error: AbstractFullDistribZkTestBase.StopableIndexingThread is not public in AbstractFullDistribZkTestBase;
cannot be accessed from outside package
    [javac]       StopableIndexingThread indexThread = new StopableIndexingThread(null, server,
"1", true, docCount);
    [javac]                                                ^
    [javac] /home/markus/src/solr/trunk/solr/core/src/test/org/apache/solr/cloud/hdfs/HdfsWriteToMultipleCollectionsTest.java:113:
error: AbstractFullDistribZkTestBase.StopableIndexingThread is not public in AbstractFullDistribZkTestBase;
cannot be accessed from outside package
    [javac]     for (StopableIndexingThread thread : threads) {
    [javac]          ^
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.
    [javac] 11 errors

{code}

> You should be able to use one pool of memory for multiple collection's HDFS block caches.
> -----------------------------------------------------------------------------------------
>
>                 Key: SOLR-5714
>                 URL: https://issues.apache.org/jira/browse/SOLR-5714
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: Mark Miller
>            Assignee: Mark Miller
>             Fix For: 4.8, 5.0
>
>         Attachments: SOLR-5714.patch, SOLR-5714.patch, SOLR-5714.patch, SOLR-5714.patch
>
>
> Currently, you have to specify how much direct memory to allocate per SolrCore. This
can be inefficient, and has some negative consequences - for instance, when replicating, many
times two HDFS directories will exist for the same index briefly, which will double the RAM
used for that SolrCore.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message