lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Diego Ceccarelli (BLOOMBERG/ LONDON)" <dceccarel...@bloomberg.net>
Subject Re: About custom score using Solr8/Lucene8
Date Thu, 09 May 2019 15:40:05 GMT
Have you seen the rerank plugin [1] ? maybe you could apply your custom scoring function only
to the top documents? 

[1]  https://lucene.apache.org/solr/6_6_0//solr-core/org/apache/solr/search/ReRankQParserPlugin.html

From: java-user@lucene.apache.org At: 05/09/19 16:24:07To:  java-user@lucene.apache.org
Subject: Re: About custom score using Solr8/Lucene8

Diego it could be a nice workaround in case i was not able to perform custom 
scoring (or in case custom scoring function would generate a slow query)
Thanks

I still hope i will be able to implement a custom score function in some way 

I was almost able to achieve such result with ValueSourceParser but i’m not 
sure is the best practice

> Il giorno 9 mag 2019, alle ore 16:02, Alessandro Benedetti 
<benedetti.alex85@gmail.com> ha scritto:
> 
> What about external fields?
> 
https://lucene.apache.org/solr/guide/6_6/working-with-external-files-and-process
es.html
> I admit I have not read in details all the thread, but from a superficial
> reading, it could help.
> Apologies if it's not helpful enough!
> 
> Cheers
> 
> 
> On Thu, May 9, 2019 at 12:57 PM Federico Pici <akirapix@libero.it.invalid>
> wrote:
> 
>> Yes this was an option, and it solves a part of the problem.
>> 
>> But let’s consider this use case:
>> 
>> for each user index we need to calculate some important informations based
>> on the most relevant terms, the “starred” documents, the number of times
>> the were opened by user (and other key factors based on documents usage).
>> So, maybe weekly, we should update this “global” factors that would affect
>> each user’s document score. If this week i opened, for example, more
>> documents of typeA and less of typeB, ALL documents of typeA would be more
>> important
>> of all documents of typeB (this is just a strategy). So each time i
>> recalculate “global” importance key factors i would need to update every
>> user’s document to change its score.
>> 
>> In this case i would fear scaling up, because as users grow up (and as
>> each user’s document number grow up) i would been updating more and more
>> document.
>> 
>> That’s why i would like to separate this moments. I would calculate
>> “global” key factors in a separate process, but i would like to use them at
>> query time to generate different scoring strate
>> 
>> 
>> 
>> 
>>> Il giorno 9 mag 2019, alle ore 13:29, Diego Ceccarelli (BLOOMBERG/
>> LONDON) <dceccarelli4@bloomberg.net> ha scritto:
>>> 
>>> why would you need to reindex all the users documents everynight? is the
>> score affected by the number of documents per user?
>>> 
>>> Also for scores, you might be able to use in-place updates:
>>> 
>>> 
>> 
https://lucene.apache.org/solr/guide/6_6/updating-parts-of-documents.html#Updati
ngPartsofDocuments-In-PlaceUpdates
>>> 
>>> so no need to reindex all the document.
>>> 
>>> 
>>> From: java-user@lucene.apache.org At: 05/09/19 11:07:37To:  Diego
>> Ceccarelli (BLOOMBERG/ LONDON ) ,  java-user@lucene.apache.org
>>> Subject: Re: About custom score using Solr8/Lucene8
>>> 
>>> Well,
>>> 
>>> my index is basically a list of documents of users. so i could say i got
>> a
>>> couple of millions of documents per user.
>>> 
>>> I need to produce a score that is unrelated with search term (something
>> like
>>> “give me documents that are the most important to user”). I would
>>> search for documents of a user without specifying a particular term.
>> Documents
>>> that are the most important for a user are calculated with strategy
>> based
>>> on particular text analysis (we still do not have a strategy).
>>> 
>>> I evaluated the idea of storing a score for each document based on an
>> external
>>> scoring strategy (a job that reindex each user’s document every night?),
>> but
>>> i do not know how could scale up (more user with many documents could
>> make this
>>> strategy problematic in the long time), so the idea was to be able
>>> to calculate this scoring (or other scoring strategy) trying to follow
>> some
>>> best-practice (and as i can see using CustomScoreQuery is the most
>> preferred
>>> way)
>>> 
>>> I was not able to try a solution because of Lucene8 missing
>> CustomScoreQuery.
>>> I made something very basic using ValueSourceParser (but is this a good
>>> practice?)
>>> 
>>> 
>>>> Il giorno 9 mag 2019, alle ore 10:50, Diego Ceccarelli (BLOOMBERG/
>> LONDON)
>>> <dceccarelli4@bloomberg.net> ha scritto:
>>>> 
>>>> Hi Federico,
>>>> 
>>>> Could you give more details on what is the score that you want to
>> produce?
>>> how big is the collection?
>>>> Is the score affected by the query? if not, would be possible to store
>> the
>>> score with the document?
>>>> 
>>>> Cheers,
>>>> Diego
>>>> 
>>>> From: java-user@lucene.apache.org At: 05/09/19 08:13:35To:
>>> java-user@lucene.apache.org
>>>> Subject: Re: About custom score using Solr8/Lucene8
>>>> 
>>>> Hello Mikhail,
>>>> yes i did. I read all the page at
>>>> https://issues.apache.org/jira/browse/LUCENE-8099
>>>> <https://issues.apache.org/jira/browse/LUCENE-8099>
>>>> 
>>>> I still do not understand where/how to start to write a class to be
>> embedded
>>> in
>>>> solr8. All
>>>> available example since 2012 where written around
>>>> CustomScoreQuery/CustomScoreProvider.
>>>> 
>>>> I really don’t know how to setup a minimal project to implement custom
>>> scoring
>>>> in solr8/lucene8 using FunctionScoreQuery
>>>> 
>>>>> Il giorno 8 mag 2019, alle ore 22:15, Mikhail Khludnev <
>> mkhl@apache.org> ha
>>>> scritto:
>>>>> 
>>>>> Hello.
>>>>> 
>>>>> Have you checked the fourth item at
>>>>> http://lucene.apache.org/core/8_0_0/MIGRATE.html ?
>>>>> 
>>>>> On Wed, May 8, 2019 at 5:46 PM Federico Pici
>> <akirapix@libero.it.invalid>
>>>>> wrote:
>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> i was trying to implement custom score inside my solr8 instance.
>>>>>> On google i found some examples, all focused on using
>> CustomScoreProvider
>>>>>> and CustomScoreQuery classes. This classes have been removed in
>>>>>> Solr8/Lucene8
>>>>>> and, from Lucene 8 changeling, i saw a developer /contributor saying
>> to
>>>>>> use FunctionScoreQuery.
>>>>>> 
>>>>>> Googling around was unsuccessful as i found no example, so i have
no
>> clue
>>>>>> from where to start from
>>>>>> 
>>>>>> I think i have a very basic need. Generate a score from field in
>>>>>> documents using logic completely unrelated with query itself.
>>>>>> 
>>>>>> The only way available (until any clue…) is using
>>>>>> org.apache.solr.search.ValueSourceParser
>>>>>> 
>>>>>> 
>>>>>> Is there any example, at least to understand how to start a minimal
>> basic
>>>>>> project?
>>>>>> 
>>>>>> Thanks
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>> 
>>>>>> 
>>>>> 
>>>>> --
>>>>> Sincerely yours
>>>>> Mikhail Khludnev
>>>> 
>>>> 
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>> 
>>> 
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>> 
>> 
> 
> -- 
> --------------------------
> 
> Benedetti Alessandro
> Visiting card - http://about.me/alessandro_benedetti
> Blog - http://alexbenedetti.blogspot.co.uk
> 
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
> 
> William Blake - Songs of Experience -1794 England


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message