lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alessandro Benedetti <benedetti.ale...@gmail.com>
Subject Re: About custom score using Solr8/Lucene8
Date Thu, 09 May 2019 14:02:41 GMT
What about external fields?
https://lucene.apache.org/solr/guide/6_6/working-with-external-files-and-processes.html
I admit I have not read in details all the thread, but from a superficial
reading, it could help.
Apologies if it's not helpful enough!

Cheers


On Thu, May 9, 2019 at 12:57 PM Federico Pici <akirapix@libero.it.invalid>
wrote:

> Yes this was an option, and it solves a part of the problem.
>
> But let’s consider this use case:
>
> for each user index we need to calculate some important informations based
> on the most relevant terms, the “starred” documents, the number of times
> the were opened by user (and other key factors based on documents usage).
> So, maybe weekly, we should update this “global” factors that would affect
> each user’s document score. If this week i opened, for example, more
> documents of typeA and less of typeB, ALL documents of typeA would be more
> important
> of all documents of typeB (this is just a strategy). So each time i
> recalculate “global” importance key factors i would need to update every
> user’s document to change its score.
>
> In this case i would fear scaling up, because as users grow up (and as
> each user’s document number grow up) i would been updating more and more
> document.
>
> That’s why i would like to separate this moments. I would calculate
> “global” key factors in a separate process, but i would like to use them at
> query time to generate different scoring strate
>
>
>
>
> > Il giorno 9 mag 2019, alle ore 13:29, Diego Ceccarelli (BLOOMBERG/
> LONDON) <dceccarelli4@bloomberg.net> ha scritto:
> >
> > why would you need to reindex all the users documents everynight? is the
> score affected by the number of documents per user?
> >
> > Also for scores, you might be able to use in-place updates:
> >
> >
> https://lucene.apache.org/solr/guide/6_6/updating-parts-of-documents.html#UpdatingPartsofDocuments-In-PlaceUpdates
> >
> > so no need to reindex all the document.
> >
> >
> > From: java-user@lucene.apache.org At: 05/09/19 11:07:37To:  Diego
> Ceccarelli (BLOOMBERG/ LONDON ) ,  java-user@lucene.apache.org
> > Subject: Re: About custom score using Solr8/Lucene8
> >
> > Well,
> >
> > my index is basically a list of documents of users. so i could say i got
> a
> > couple of millions of documents per user.
> >
> > I need to produce a score that is unrelated with search term (something
> like
> > “give me documents that are the most important to user”). I would
> > search for documents of a user without specifying a particular term.
> Documents
> > that are the most important for a user are calculated with strategy
> based
> > on particular text analysis (we still do not have a strategy).
> >
> > I evaluated the idea of storing a score for each document based on an
> external
> > scoring strategy (a job that reindex each user’s document every night?),
> but
> > i do not know how could scale up (more user with many documents could
> make this
> > strategy problematic in the long time), so the idea was to be able
> > to calculate this scoring (or other scoring strategy) trying to follow
> some
> > best-practice (and as i can see using CustomScoreQuery is the most
> preferred
> > way)
> >
> > I was not able to try a solution because of Lucene8 missing
> CustomScoreQuery.
> > I made something very basic using ValueSourceParser (but is this a good
> > practice?)
> >
> >
> >> Il giorno 9 mag 2019, alle ore 10:50, Diego Ceccarelli (BLOOMBERG/
> LONDON)
> > <dceccarelli4@bloomberg.net> ha scritto:
> >>
> >> Hi Federico,
> >>
> >> Could you give more details on what is the score that you want to
> produce?
> > how big is the collection?
> >> Is the score affected by the query? if not, would be possible to store
> the
> > score with the document?
> >>
> >> Cheers,
> >> Diego
> >>
> >> From: java-user@lucene.apache.org At: 05/09/19 08:13:35To:
> > java-user@lucene.apache.org
> >> Subject: Re: About custom score using Solr8/Lucene8
> >>
> >> Hello Mikhail,
> >> yes i did. I read all the page at
> >> https://issues.apache.org/jira/browse/LUCENE-8099
> >> <https://issues.apache.org/jira/browse/LUCENE-8099>
> >>
> >> I still do not understand where/how to start to write a class to be
> embedded
> > in
> >> solr8. All
> >> available example since 2012 where written around
> >> CustomScoreQuery/CustomScoreProvider.
> >>
> >> I really don’t know how to setup a minimal project to implement custom
> > scoring
> >> in solr8/lucene8 using FunctionScoreQuery
> >>
> >>> Il giorno 8 mag 2019, alle ore 22:15, Mikhail Khludnev <
> mkhl@apache.org> ha
> >> scritto:
> >>>
> >>> Hello.
> >>>
> >>> Have you checked the fourth item at
> >>> http://lucene.apache.org/core/8_0_0/MIGRATE.html ?
> >>>
> >>> On Wed, May 8, 2019 at 5:46 PM Federico Pici
> <akirapix@libero.it.invalid>
> >>> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> i was trying to implement custom score inside my solr8 instance.
> >>>> On google i found some examples, all focused on using
> CustomScoreProvider
> >>>> and CustomScoreQuery classes. This classes have been removed in
> >>>> Solr8/Lucene8
> >>>> and, from Lucene 8 changeling, i saw a developer /contributor saying
> to
> >>>> use FunctionScoreQuery.
> >>>>
> >>>> Googling around was unsuccessful as i found no example, so i have no
> clue
> >>>> from where to start from
> >>>>
> >>>> I think i have a very basic need. Generate a score from field in
> >>>> documents using logic completely unrelated with query itself.
> >>>>
> >>>> The only way available (until any clue…) is using
> >>>> org.apache.solr.search.ValueSourceParser
> >>>>
> >>>>
> >>>> Is there any example, at least to understand how to start a minimal
> basic
> >>>> project?
> >>>>
> >>>> Thanks
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >>>> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>>>
> >>>>
> >>>
> >>> --
> >>> Sincerely yours
> >>> Mikhail Khludnev
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

-- 
--------------------------

Benedetti Alessandro
Visiting card - http://about.me/alessandro_benedetti
Blog - http://alexbenedetti.blogspot.co.uk

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message