lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Solr MapReduce Indexer Tool is failing for empty core name.
Date Mon, 02 Jan 2017 18:13:44 GMT
MRIT has never worked by specifying cores on the command line,
it's always been about collections. It reads the data from ZK and then
tries to create temporary cores to index into. Eventually, these
temporary cores are merged into the final index for the shard.

So a couple of wild shots in the dark.

- there was an issue a while ago about naming conventions. While the
period in your collection name _should_ be OK, what happens if you
use a name without a period?

- It's actually not recommended to just try to plop 4x configs into 6x
Solr's, rather start with 6x configs and add your modifications. In particular,
if you still have 4x as your luceneMatchVersion I don't quite know what
happens. It's vaguely possible that this error is what's bubbled back up
due to something wonky like this. This is unlikely frankly since you say that
all the rest of your operations are OK, but maybe worth trying.

- Failing all that I wonder if you have some old jars in your classpath.

- Finally, I'd try to start with a fresh dummy collection and do the simplest
config I could and build up from there.

Best,
Erick



On Sun, Jan 1, 2017 at 11:51 PM, Manan Sheth <manan.sheth@impetus.co.in> wrote:
> Hi All,
>
> Please help me out if anyone has executed solr map reduce indexer tool with solr 6. This
is still failing and no hints for the error shown in below mail thread.
>
> Thanks,
> Manan Sheth
> ________________________________________
> From: Manan Sheth
> Sent: Friday, December 16, 2016 2:52 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr MapReduce Indexer Tool is failing for empty core name.
>
> Thats what I presume and it should start utilizing the collection only. The collection
param has already been specified and it should take all details from there only. also, core
to collection change was happed in solr 4. The map reduce inderexer for solr 4.10 is working
correctly with this, but not for solr 6.
> ________________________________________
> From: Reth RM <reth.iksam@gmail.com>
> Sent: Friday, December 16, 2016 12:45 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr MapReduce Indexer Tool is failing for empty core name.
>
> The primary difference has been solr to solr-cloud in later version,
> starting from solr4.0  And what happens if you try starting solr in stand
> alone mode, solr cloud does not consider 'core' anymore, it considers
> 'collection' as param.
>
>
> On Thu, Dec 15, 2016 at 11:05 PM, Manan Sheth <manan.sheth@impetus.co.in>
> wrote:
>
>> Thanks Reth. As noted this is the same map reduce based indexer tool that
>> comes shipped with the solr distribution by default.
>>
>> It only take the zk_host details and extracts all required information
>> from there only. It does not have core specific configurations. The same
>> tool released with solr 4.10 distro is working correctly, it seems to be
>> some issue/ changes from solr 5 onwards. I have tested it for both solr 5.5
>> & solr 6.2.1 and the behaviour remains same for both.
>>
>> Thanks,
>> Manan Sheth
>> ________________________________________
>> From: Reth RM <reth.iksam@gmail.com>
>> Sent: Friday, December 16, 2016 12:21 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Solr MapReduce Indexer Tool is failing for empty core name.
>>
>> It looks like command line tool that you are using to initiate index
>> process,  is expecting some name to solr-core with respective command line
>> param. use -help on the command line tool that you are using and check the
>> solr-core-name parameter key, pass that also with some value.
>>
>>
>> On Tue, Dec 13, 2016 at 5:44 AM, Manan Sheth <manan.sheth@impetus.co.in>
>> wrote:
>>
>> > Hi All,
>> >
>> >
>> > While working on a migration project from Solr 4 to Solr 6, I need to
>> > reindex my data using Solr map reduce Indexer tool in offline mode with
>> > avro data.
>> >
>> > While executing the map reduce indexer tool shipped with solr 6.2.1, it
>> is
>> > throwing error of cannot create core with empty name value. The solr
>> > instances are running fine with new indexed are being added and modified
>> > correctly. Below is the command that was being fired:
>> >
>> >
>> > hadoop --config /etc/hadoop/conf jar /home/impadmin/solr-6.2.1/
>> dist/solr-map-reduce-*.jar
>> > -D 'mapred.child.java.opts=-Xmx500m' \
>> >        -libjars `echo /home/impadmin/solr6lib/*.jar | sed 's/ /,/g'`
>> > --morphline-file /home/impadmin/app_quotes_morphline_actual.conf \
>> >        --zk-host 172.26.45.71:9984 --output-dir hdfs://
>> > impetus-i0056.impetus.co.in:8020/user/impadmin/
>> > MapReduceIndexerTool/output5 \
>> >        --collection app.quotes --log4j src/test/resources/log4j.
>> properties
>> > --verbose \
>> >          "hdfs://impetus-i0056.impetus.co.in:8020/user/impadmin/
>> > MapReduceIndexerTool/5d63e0f8-afc1-483e-bd3f-d508c885d794-000000"
>> >
>> >
>> > Below is the complete snapshot of error trace:
>> >
>> >
>> > Failed to initialize record writer for org.apache.solr.hadoop.
>> > MapReduceIndexerTool/MorphlineMapper, attempt_1479795440861_0343_r_
>> > 000000_0
>> >     at org.apache.solr.hadoop.SolrRecordWriter.<init>(
>> > SolrRecordWriter.java:128)
>> >     at org.apache.solr.hadoop.SolrOutputFormat.getRecordWriter(
>> > SolrOutputFormat.java:163)
>> >     at org.apache.hadoop.mapred.ReduceTask$
>> NewTrackingRecordWriter.<init>
>> > (ReduceTask.java:540)
>> >     at org.apache.hadoop.mapred.ReduceTask.runNewReducer(
>> > ReduceTask.java:614)
>> >     at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
>> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
>> >     at java.security.AccessController.doPrivileged(Native Method)
>> >     at javax.security.auth.Subject.doAs(Subject.java:422)
>> >     at org.apache.hadoop.security.UserGroupInformation.doAs(
>> > UserGroupInformation.java:1709)
>> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
>> > Caused by: org.apache.solr.common.SolrException: Cannot create core with
>> > empty name value
>> >     at org.apache.solr.core.CoreDescriptor.checkPropertyIsNotEmpty(
>> > CoreDescriptor.java:280)
>> >     at org.apache.solr.core.CoreDescriptor.<init>(
>> CoreDescriptor.java:191)
>> >     at org.apache.solr.core.CoreContainer.create(CoreContainer.java:754)
>> >     at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)
>> >     at org.apache.solr.hadoop.SolrRecordWriter.createEmbeddedSolrServer(
>> > SolrRecordWriter.java:163)
>> >     at org.apache.solr.hadoop.SolrRecordWriter.<init>(
>> SolrRecordWriter.java:121)
>> > ... 9 more
>> >
>> > Additional points to note:
>> >
>> >
>> >   *   The solrconfig and schema files are copied as is from Solr 4.
>> >   *   Once collection is deployed, user can perform all operations on the
>> > collection without any issue.
>> >   *   The indexation process is working fine with the same tool on Solr
>> 4.
>> >
>> > Please help.
>> >
>> >
>> > Thanks,
>> >
>> > Manan Sheth
>> >
>> > ________________________________
>> >
>> >
>> >
>> >
>> >
>> >
>> > NOTE: This message may contain information that is confidential,
>> > proprietary, privileged or otherwise protected by law. The message is
>> > intended solely for the named addressee. If received in error, please
>> > destroy and notify the sender. Any use of this email is prohibited when
>> > received in error. Impetus does not represent, warrant and/or guarantee,
>> > that the integrity of this communication has been maintained nor that the
>> > communication is free of errors, virus, interception or interference.
>> >
>>
>> ________________________________
>>
>>
>>
>>
>>
>>
>> NOTE: This message may contain information that is confidential,
>> proprietary, privileged or otherwise protected by law. The message is
>> intended solely for the named addressee. If received in error, please
>> destroy and notify the sender. Any use of this email is prohibited when
>> received in error. Impetus does not represent, warrant and/or guarantee,
>> that the integrity of this communication has been maintained nor that the
>> communication is free of errors, virus, interception or interference.
>>
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged
or otherwise protected by law. The message is intended solely for the named addressee. If
received in error, please destroy and notify the sender. Any use of this email is prohibited
when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity
of this communication has been maintained nor that the communication is free of errors, virus,
interception or interference.

Mime
View raw message