lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gopal Patwa <gopalpa...@gmail.com>
Subject Re: Identify specific document insert error inside a solrj batch request
Date Sun, 03 Aug 2014 18:22:20 GMT
for reference implementation to Sanitize Unknown SolrFields, you can see
below link

https://github.com/cloudera/cdk/blob/master/cdk-morphlines/cdk-morphlines-solr-core/src/main/java/com/cloudera/cdk/morphline/solr/SanitizeUnknownSolrFieldsBuilder.java



On Sat, Aug 2, 2014 at 8:24 PM, Umesh Prasad <umesh.iitk@gmail.com> wrote:

> Solr  schema over REST https://wiki.apache.org/solr/SchemaRESTAPI
>
> https://cwiki.apache.org/confluence/display/solr/Schema+API
>
> You can use that for getting required fields and validate at client side ..
>
>
>
>
>
> On 31 July 2014 14:32, Liram Vardi <liramv@checkpoint.com> wrote:
>
> > Hi Jack,
> > Thank you for your reply.
> > This is the Solr stack trace. As you can see, the missing field is
> > "hourOfDay".
> >
> > Thanks,
> > Liram
> >
> > 2014-07-30 14:27:54,934 ERROR [qtp-608368492-19] (SolrException.java:108)
> > - org.apache.solr.common.SolrException:
> > [doc=53b16126-0000-0002-2b03-17ac4d4a07b6] missing required field:
> hourOfDay
> >         at
> >
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:189)
> >         at
> >
> org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:73)
> >         at
> >
> org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:210)
> >         at
> >
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
> >         at
> >
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
> >         at
> >
> org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:556)
> >         at
> >
> org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:692)
> >         at
> >
> org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435)
> >         at
> >
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
> >         at
> >
> org.apache.solr.update.processor.AbstractDefaultValueUpdateProcessorFactory$DefaultValueUpdateProcessor.processAdd(AbstractDefaultValueUpdateProcessorFactory.java:94)
> >         at
> >
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
> >         at
> >
> com.checkpoint.solr_plugins.MulticoreUpdateRequestProcessor.processAdd(MulticoreUpdateRequestProcessorFactory.java:152)
> >         at
> >
> org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:246)
> >         at
> > org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:173)
> >         at
> >
> org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
> >         at
> >
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
> >         at
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> >         at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904)
> >         at
> >
> com.checkpoint.solr_plugins.MulticoreUpdateRequestProcessor.processAdd(MulticoreUpdateRequestProcessorFactory.java:248)
> >         at
> >
> org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:86)
> >         at
> >
> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:143)
> >         at
> >
> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:123)
> >         at
> > org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:220)
> >         at
> >
> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:108)
> >         at
> > org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:185)
> >         at
> > org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:111)
> >         at
> >
> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:150)
> >         at
> >
> org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:96)
> >         at
> > org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:55)
> >         at
> >
> org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
> >         at
> >
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
> >         at
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> >         at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904)
> >         at
> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:659)
> >         at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:362)
> >         at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158)
> >         at
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474)
> >         at
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
> >         at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
> >         at
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
> >         at
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> >         at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
> >         at
> > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
> >         at
> >
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> >         at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
> >         at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> >         at
> >
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
> >         at
> >
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
> >         at
> >
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
> >         at org.eclipse.jetty.server.Server.handle(Server.java:370)
> >         at
> >
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
> >         at
> >
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
> >         at
> >
> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:960)
> >         at
> >
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1021)
> >         at
> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:957)
> >
> > -----Original Message-----
> > From: Jack Krupansky [mailto:jack@basetechnology.com]
> > Sent: Wednesday, July 30, 2014 5:53 PM
> > To: solr-user@lucene.apache.org
> > Subject: Re: Identify specific document insert error inside a solrj batch
> > request
> >
> > Agreed that this is a problem with Solr. If it was merely "bad input",
> > Solr should be returning a 4xx error.
> >
> > I don't know if we already have a Jira for this. If not, one should be
> > filed.
> >
> > There are two issues:
> >
> > 1. The status code should be 4xx with an appropriate message about "bad
> > input".
> >
> > 2. The offset of the offending document should be reported so that the
> app
> > can locate the problem to resolve it.
> >
> > Give us the actual server stack trace so we can verify whether this was
> > simply "user error" or some defect in Solr itself.
> >
> > -- Jack Krupansky
> >
> > -----Original Message-----
> > From: Liram Vardi
> > Sent: Wednesday, July 30, 2014 9:25 AM
> > To: solr-user@lucene.apache.org
> > Subject: Identify specific document insert error inside a solrj batch
> > request
> >
> > Hi All,
> >
> > I have a question regarding the use of HttpSolrServer (SolrJ).
> > I have a collection of SolrInputDocuments I want to send to Solr as a
> > batch.
> > Now, let's assume that one of the docs inside this collection is
> corrupted
> > (missing some "required" field).
> > When I send the batch of docs to solr using
> HttpSolrServer.add(Collection<
> > SolrInputDocument> docs) I am getting the following general exception:
> >
> > "org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
> > Server at http://172.23.3.91:8210/solr/template returned non ok
> > status:500,
> > message:Server Error"
> >
> > When I check Solr log, I can identify exactly which is the corrupted
> > document.
> >
> > My question:
> > Is it possible to identify the problematic document at the client side?
> > (for
> > recovery purposes)
> >
> > Thanks,
> > Liram
> >
> >
> > Email secured by Check Point
> >
>
>
>
> --
> ---
> Thanks & Regards
> Umesh Prasad
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message