lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Norskog <goks...@gmail.com>
Subject Re: Solr date and string search problem
Date Mon, 16 Nov 2009 23:57:07 GMT
If you search a 'date' field you have to give it a correct UTC formatted string.

You can have a copy of the date field into a string field, and search
the string. These will have to be pure UTC formatted strings also. The
<copyField> directive can do this copying for you. The target field
does not have to be stored, only indexed.

On Sun, Nov 15, 2009 at 9:56 PM, ashokcz <ashokkumar.ganesan@tcs.com> wrote:
>
> Hi Lance Norskog ,
> Thanks for your reply.
>
> Let me first put the config files details.
> These are the fields i have defined
>
>  <fieldType class="solr.TextField" name="alphaOnlySort" omitNorms="true"
> sortMissingLast="true">
>      <analyzer type="query">
>        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>        <filter catenateAll="0" catenateNumbers="0" catenateWords="0"
> class="solr.WordDelimiterFilterFactory" generateNumberParts="1"
> generateWordParts="1"/>
>        <filter class="solr.LowerCaseFilterFactory"/>
>        <filter class="solr.TrimFilterFactory"/>
>      </analyzer>
>    </fieldType>
>
>
>    <fieldType class="solr.TextField" name="alphaOnlySortFacet"
> omitNorms="true" sortMissingLast="true">
>      <analyzer type="query">
>        <tokenizer class="solr.KeywordTokenizerFactory"/>
>        <filter class="solr.LowerCaseFilterFactory"/>
>        <filter class="solr.TrimFilterFactory"/>
>      </analyzer>
>    </fieldType>
>
>    <fieldType class="solr.TextField" name="specialFacet" omitNorms="true"
> sortMissingLast="true">
>      <analyzer type="query">
>        <tokenizer class="solr.CommaTokenizerFactory"/>
>        <filter class="solr.LowerCaseFilterFactory"/>
>        <filter class="solr.TrimFilterFactory"/>
>      </analyzer>
>    </fieldType>
>  </types>
>
>
> <field indexed="true" multiValued="true" name="text" stored="false"
> type="text"/>
> <field indexed="true" name="id" required="true" stored="true"
> type="string"/>
> <field indexed="true" name="status" stored="true"
> type="alphaOnlySortFacet"/>
> <field indexed="false" name="noofViews" stored="true" type="integer"/>
> <field indexed="true" name="uploadedBy" stored="true" type="text"/>
> <field indexed="true" name="uploadedOn" stored="true" type="date"/>
> <field indexed="true" name="popularity" stored="true" type="float"/>
> <field indexed="true" name="Plant" stored="true" type="specialFacet"/>
> <field indexed="true" name="PlantSearch" stored="true"
> type="alphaOnlySort"/>
> <field indexed="true" name="Geography" stored="true"
> type="alphaOnlySortFacet"/>
> <field indexed="true" name="GeographySearch" stored="true"
> type="alphaOnlySort"/>
> <field indexed="true" name="Region" stored="true"
> type="alphaOnlySortFacet"/>
> <field indexed="true" name="RegionSearch" stored="true"
> type="alphaOnlySort"/>
> <field indexed="true" name="Country" stored="true"
> type="alphaOnlySortFacet"/>
> <field indexed="true" name="CountrySearch" stored="true"
> type="alphaOnlySort"/>
> <field indexed="true" name="BusUnit" stored="true" type="specialFacet"/>
> <field indexed="true" name="BusUnitSearch" stored="true"
> type="alphaOnlySort"/>
> <field indexed="true" name="BusinessFunction" stored="true"
> type="alphaOnlySortFacet"/>
> <field indexed="true" name="BusinessFunctionSearch" stored="true"
> type="alphaOnlySort"/>
> <field indexed="true" name="Functionality" stored="true"
> type="alphaOnlySortFacet"/>
> <field indexed="true" name="FunctionalitySearch" stored="true"
> type="alphaOnlySort"/>
> <field indexed="true" name="Businessprocesses" stored="true" type="text"/>
> <field indexed="true" name="UploadedDate" stored="true" type="date"/>
>
>
> and this is my requestHandler configuration
>
>
> <requestHandler class="solr.DisMaxRequestHandler" name="dismaxRelAndPop">
>    <lst name="defaults">
>     <str name="echoParams">explicit</str>
>     <float name="tie">0.01</float>
>     <str name="qf">PlantSearch^1 GeographySearch^1 RegionSearch^1
> CountrySearch^1 BusUnitSearch^1 BusinessFunctionSearch^1
> Businessprocesses^1 LifecycleStatus^1 ApplicationNature^1 UploadedDate^1
> </str>
>     <str name="pf">PlantSearch^1 GeographySearch^1 RegionSearch^1
> CountrySearch^1 BusUnitSearch^1 BusinessFunctionSearch^1
> Businessprocesses^1 LifecycleStatus^1 ApplicationNature^1 UploadedDate^1
> </str>
>     <str name="fl">*,score</str>
>     <str name="bf">
>        ord(popularity)^0.5 recip(rord(popularity),1,1000,1000)^0.3
>     </str>
>     <str name="q.alt">*:*</str>
>     <str name="mm">
>        10&lt;50%
>     </str>
>    </lst>
>  </requestHandler>
>
>
>  and this is the query thats been fired.
>
>
>
> facet.limit=-1&rows=10&start=0&facet=true&facet.mincount=1&facet.field=Geography&facet.field=Country&facet.field=Functionality&facet.field=BusinessFunction&facet.field=BusUnit&facet.field=Region&facet.field=PGServiceManager&facet.field=AppName&facet.field=Plant&facet.field=status&q=Behavior&facet.sort=true
>
>  i clearly understand where the problem is happening , but dont know how to
> resolve it .
>
>  i have defined UploadedDate as date field and i have defined in my request
> handler to search in UploadedDate field also  (  .... UploadedDate^1 .....)
>
>  but what happens is every query that is been fired is converted to date
> and it throws me an error.
>  if i remove UploadedDate from  request handler it works fine.
>
>  so i dont know how to have some tring fields and some date fields together
> co exist in a request handler  ??
>  and according to the given query solr should filter it out in all the
> fields and should give me the result back .
>  is there an way to do tat??
>  sorry for a such a long repsone :)
>
>  thanks
>  -----------
>  Ashok
>
>
>
>
>
>
> Lance Norskog-2 wrote:
>>
>> This line is the key:
>>> SEVERE: org.apache.solr.core.SolrException: Invalid Date
>>> String:'Behavior'
>>>        at org.apache.solr.schema.DateField.toInternal(DateField.java:108)
>>>        at
>>
>> The string 'Behavior' is being parsed as a date, and fails. Your query
>> is attempting to find this as a date. Please post your query. and the
>> <requestHandler> configuration that it is used against.
>>
>> On Sat, Nov 14, 2009 at 4:16 AM, ashokcz <ashokkumar.ganesan@tcs.com>
>> wrote:
>>>
>>> Hi ,
>>> I have been using solr1.2 for a year and now i m facing a weird problem.
>>> Till now i have used only string and number solr types for the search
>>> field
>>> .
>>> and whatever string the users are trying to search will pass it on to the
>>> search engine and it will find in appropraite fields and return me the
>>> results.
>>>
>>> But now i have added a another field with type DATE and made it as a
>>> search
>>> field.
>>> so what happens is whatever string that i m giving to solr tries to
>>> convert
>>> to date and throws me an error.
>>>
>>>  Nov 14, 2009 4:36:05 PM org.apache.solr.core.SolrException log
>>> SEVERE: org.apache.solr.core.SolrException: Invalid Date
>>> String:'Behavior'
>>>        at org.apache.solr.schema.DateField.toInternal(DateField.java:108)
>>>        at
>>> org.apache.solr.schema.FieldType$DefaultAnalyzer$1.next(FieldType.java:298)
>>>        at
>>> org.apache.lucene.queryParser.QueryParser.getFieldQuery(QueryParser.java:437)
>>>        at
>>> org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java:78)
>>>        at
>>> org.apache.solr.util.SolrPluginUtils$DisjunctionMaxQueryParser.getFieldQuery(SolrPluginUtils.java:774)
>>>        at
>>> org.apache.solr.util.SolrPluginUtils$DisjunctionMaxQueryParser.getFieldQuery(SolrPluginUtils.java:762)
>>>        at
>>> org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1092)
>>>        at
>>> org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:979)
>>>        at
>>> org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:907)
>>>        at
>>> org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:896)
>>>        at
>>> org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:146)
>>>        at
>>> org.apache.solr.request.DisMaxRequestHandler.handleRequestBody(DisMaxRequestHandler.java:238)
>>>        at
>>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:77)
>>>        at org.apache.solr.core.SolrCore.execute(SolrCore.java:706)
>>>        at
>>> com.tcs.mighty.cb.service.CoreEntitySearcherImpl.solrSearch(CoreEntitySearcherImpl.java:896)
>>>        at
>>> com.tcs.mighty.cb.service.CoreEntitySearcherImpl.search(CoreEntitySearcherImpl.java:342)
>>>        at
>>> com.tcs.mighty.cb.service.CoreEntitySearcherImpl.handleGetSearchResults(CoreEntitySearcherImpl.java:52)
>>>        at
>>> com.tcs.mighty.cb.service.CoreEntitySearcherBase.getSearchResults(CoreEntitySearcherBase.java:122)
>>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>        at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>        at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>        at java.lang.reflect.Method.invoke(Method.java:585)
>>>        at
>>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:287)
>>>        at
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:181)
>>>        at
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:148)
>>>        at
>>> org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:97)
>>>        at
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:170)
>>>        at
>>> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
>>>        at
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:170)
>>>        at
>>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:176)
>>>        at $Proxy141.getSearchResults(Unknown Source)
>>>        at
>>> com.tcs.mighty.cb.service.ejb.CoreEntitySearcherBean.getSearchResults(CoreEntitySearcherBean.java:24)
>>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>        at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>        at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>        at java.lang.reflect.Method.invoke(Method.java:585)
>>>        at
>>> org.jboss.ejb.StatelessSessionContainer$ContainerInterceptor.invoke(StatelessSessionContainer.java:683)
>>>        at
>>> org.jboss.resource.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:186)
>>>        at
>>> org.jboss.ejb.plugins.StatelessSessionInstanceInterceptor.invoke(StatelessSessionInstanceInterceptor.java:72)
>>>        at
>>> org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:84)
>>>        at
>>> org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:315)
>>>        at
>>> org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:148)
>>>        at
>>> org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:111)
>>>        at
>>> org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:191)
>>>        at
>>> org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:122)
>>>        at
>>> org.jboss.ejb.StatelessSessionContainer.internalInvoke(StatelessSessionContainer.java:331)
>>>        at org.jboss.ejb.Container.invoke(Container.java:709)
>>>        at
>>> org.jboss.ejb.plugins.local.BaseLocalProxyFactory.invoke(BaseLocalProxyFactory.java:419)
>>>        at
>>> org.jboss.ejb.plugins.local.StatelessSessionProxy.invoke(StatelessSessionProxy.java:83)
>>>        at $Proxy96.getSearchResults(Unknown Source)
>>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>        at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>        at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>        at java.lang.reflect.Method.invoke(Method.java:585)
>>>        at
>>> org.springframework.ejb.access.LocalSlsbInvokerInterceptor.invoke(LocalSlsbInvokerInterceptor.java:71)
>>>        at
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:170)
>>>        at
>>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:176)
>>>        at $Proxy141.getSearchResults(Unknown Source)
>>>        at
>>> com.tcs.mighty.cb.web.search.SearchControllerImpl.getSearchResults(SearchControllerImpl.java:71)
>>>        at
>>> com.tcs.mighty.cb.web.search.Search.__getSearchResults(Search.java:96)
>>>        at
>>> com.tcs.mighty.cb.web.search.Search._searchController(Search.java:87)
>>>        at com.tcs.mighty.cb.web.search.Search.execute(Search.java:35)
>>>        at
>>> org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:421)
>>>        at
>>> org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:226)
>>>        at
>>> org.apache.struts.action.ActionServlet.process(ActionServlet.java:1164)
>>>        at
>>> org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:415)
>>>        at
>>> org.andromda.presentation.bpm4struts.ActionServlet.doPost(ActionServlet.java:48)
>>>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>>>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
>>>        at
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
>>>        at
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157)
>>>        at
>>> org.displaytag.filter.ResponseOverrideFilter.doFilter(ResponseOverrideFilter.java:125)
>>>        at
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:186)
>>>        at
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157)
>>>        at
>>> org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:75)
>>>        at
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:186)
>>>        at
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157)
>>>        at
>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:214)
>>>        at
>>> org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
>>>        at
>>> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
>>>        at
>>> org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198)
>>>        at
>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152)
>>>        at
>>> org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
>>>        at
>>> org.jboss.web.tomcat.security.CustomPrincipalValve.invoke(CustomPrincipalValve.java:66)
>>>        at
>>> org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
>>>        at
>>> org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:158)
>>>        at
>>> org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
>>>        at
>>> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
>>>        at
>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
>>>        at
>>> org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
>>>        at
>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)
>>>        at
>>> org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.jav
>>> 16:36:05,117 INFO  [STDOUT] a:102)
>>>        at
>>> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
>>>        at
>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>>>        at
>>> org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
>>>        at
>>> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
>>>        at
>>> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
>>>        at
>>> org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
>>>        at
>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)
>>>        at
>>> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)
>>>        at
>>> org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
>>>        at
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683)
>>>        at java.lang.Thread.run(Thread.java:595)
>>>
>>>
>>> I find its a weird thing to happen or am i not doing it properly ??
>>> can someone pls suggest me something ??
>>>
>>> Thanks
>>> --------------------
>>> Ashok
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/Solr-date-and-string-search-problem-tp26349545p26349545.html
>>> Sent from the Solr - User mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Lance Norskog
>> goksron@gmail.com
>>
>>
>
> --
> View this message in context: http://old.nabble.com/Solr-date-and-string-search-problem-tp26349545p26367234.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>



-- 
Lance Norskog
goksron@gmail.com

Mime
View raw message