lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bernd Fehling <bernd.fehl...@uni-bielefeld.de>
Subject Re: Solr Size Limitation upto 32 KB files
Date Wed, 02 Jan 2019 12:03:00 GMT
Hi,
I don't know the limits about Solr 4.2.1 but the RefGuide of Solr 6.6
says about Field Types for Class StrField:
"String (UTF-8 encoded string or Unicode). Strings are intended for
small fields and are not tokenized or analyzed in any way.
They have a hard limit of slightly less than 32K."

If you are trying to add larger content then you have to "chop" that
by yourself and add it as multivalued. Can be done within a self written loader.

Don't forget, Solr/Lucene is an indexer and not a fulltext engine.

Regards
Bernd


Am 02.01.19 um 10:23 schrieb Kranthi Kumar K:
> Hi,
> 
> We are currently using Solr 4.2.1 version in our project and everything is going well.
But recently, we are facing an issue with Solr Data Import. It is not importing the files
with size greater than 32766 bytes (i.e, 32 kb) and showing 2 exceptions:
> 
> 
>    1.  java.lang.illegalargumentexception
>    2.  org.apache.lucene.util.bytesref hash$maxbyteslengthexceededexception
> 
> 
> Please find the attached screenshot for reference.
> 
> We have searched for solutions in many forums and didn't find the exact solution for
this issue. Interestingly, we found in the article, by changing the type of the 'field' from
sting to  'text_general' might solve the issue. Please have a look in the below forum:
> 
> https://stackoverflow.com/questions/29445323/adding-a-document-to-the-index-in-solr-document-contains-at-least-one-immense-t
> 
> Schema.xml:
> Changed from:
> '<field name="text" type="string_rev" indexed="true" stored="false" multiValued="true"
/>'
> 
> Changed to:
> '<field name="text" type="text_general " indexed="true" stored="false" multiValued="true"
/>'
> 
> We have tried it but still it is not importing the files > 32 KB or 32766 bytes.
> 
> Could you please let us know the solution to fix this issue? We'll be awaiting your reply.
> 
> 
> [image001]
> Thanks & Regards,
> Kranthi Kumar.K,
> Software Engineer,
> Ccube Fintech Global Services Pvt Ltd.,
> Email/Skype: kranthikumar.k@ccubefintech.com<mailto:kranthikumar.k@ccubefintech.com>,
> Mobile: +91-8978078449.
> 
> 
> 

Mime
View raw message