lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (JIRA)" <>
Subject [jira] [Commented] (SOLR-12903) Query Source Tracker custom Solr component
Date Wed, 16 Jan 2019 18:55:00 GMT


Hoss Man commented on SOLR-12903:

So, my first tought when reviewing this patch is that the entire "qi" param concept (with
the configure list of allowed "queryIdentifiers" and the "failQueries" option to reject requests
if the "qi" param isn't specified and allowed) really just seems like an inferior version
of the the existing Authn/Authz Plugin support in solr...


...which has the existing RuleBasedAuthorizationPlugin that can reject/allow access to specific
request handlers based on the user (ie: "qi") – with the added bonus that the users will
be authenticated first. (but if there's a particular use case where trued authentication isn't
needed, then use use the empty string as the password for all users. (or I suppose a solr
cluster _could_ use a custom {{AuthenticationPlugin}} that just trusted any request with a
"qi" param and used that as the user name w/o requiring any password/credentials)

But once you take the "authorizing user access to a /path" out of the picture since we already
have a good solution for that, what's left is a really interesting bit: "recording per user
metrics for /path" ... i think that makes a lot of sense as an optional SearchComponent. It
may not scale well if the number of unique users is astronomical (ie: forwarded kerberos credenials
from a front end application) so it should definitely still be an optional component, but
for many cases – particularly where client applications each use unique credentials to talk
to the solr cluster – it would make a lot of sense.

The one hitch is that the way metrics are _typical_ "created" is the way the patch currently
does things: on init, created a MetricsRegistry with the list of pre-defined/hardcoded Metrics
you plan on using in your code. But in order for this to be general enough to support per-user
metrics (w/o making hardcoded assumptions about the list of users) that won't work.

I asked [~ab] about this offline, and he pointed out that we already have a solution for this
type of situation in Solr via the {{MetricsMap implements Gauge<Map<String,Object>>}}
class. We use this in places like {{SolrFieldCacheBean}} where the set of metrics can change
over time as solr is running, and {{MetricUtils}} has helper functions for unwinding &
filtering compound/sub metrics (ie: exposing Meters in via MetricsMap). He suggested using
a SolrCache as a backing store for the MetricsMap – which would also help limit the risk
of an explosion of users (the the choice between and LRU cache or an LFU cache would let solr
admins control whether they care more about the metrics for the recently active users or the
most frequently active users if/when an explosion did occur)
So I think what would make sense is:
 * Rename this something like AuthenticatedUserMetricsComponent
 * Make the only required configuration option be the name of an already configured user defined
 ** our docs should strongly encourage using NoOpRegenerator
 * register a MetricsMap with an initializer backed by that cache
 * Rip out all the "qi" logic and replace it with code to just use {{SolrQueryRequest.getUserPrincipal()}}
 * use the configured cache to track the username => Meter mapping and record the request.

> Query Source Tracker custom Solr component
> ------------------------------------------
>                 Key: SOLR-12903
>                 URL:
>             Project: Solr
>          Issue Type: Task
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Tirth Rajen Mehta
>            Priority: Minor
>          Time Spent: 10m
>  Remaining Estimate: 0h
> Added a Query Source Tracker custom Solr component (
>  * This component can be configured for a RequestHandler for query requests.
>  * This component mandates that clients to pass in a "qi" request parameter with a valid
value which is configured in the SearchComponent definition in the solrconfig.xml file.
>  * It fails the query if the "qi" parameter is missing or if the value passed in is in-valid.
This behavior of failing the queries can be controlled by the failQueries config parameter.
>  * It also collects the rate per sec metric per unique "qi" value.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message