lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shawn Heisey (JIRA)" <>
Subject [jira] [Commented] (LUCENE-7960) NGram filters -- add option to keep short terms
Date Thu, 03 May 2018 00:19:00 GMT


Shawn Heisey commented on LUCENE-7960:

An example of where I used the ellipsis notation in my own code to make a boolean argument

	 * Fully close a connection, statement, and result set, ignoring any errors
	 * that occur. Any of the three resources here can be null, but at least one
	 * of them must NOT be null.
	 * @param rs the ResultSet to close.
	 * @param st the Statement to close.
	 * @param conn The Connection to close.
	 * @param forceFlags This odd ellipsis parameter is used for one thing
	 *            currently: a flag to indicate whether or not a close will be
	 *            forced on all provided resources even if everything doesn't
	 *            match up. In theory, a statement derived from the resultset
	 *            and connections derived from either one should be exactly the
	 *            same object as the ones provided to the method. If the flag is
	 *            false, then only the first non-null resource provided and any
	 *            parent resources derived from that resource will be closed. If
	 *            it is true, ALL resources including derived resources will be
	 *            closed. Mismatches will be logged either way. The ellipsis
	 *            notation is so that this parameter is optional. If omitted, it
	 *            will default to false.
	 * @throws IllegalArgumentException if all three resources are null.
	public static void fullQuietClose(ResultSet rs, Statement st, Connection conn,
			boolean... forceFlags)

> NGram filters -- add option to keep short terms
> -----------------------------------------------
>                 Key: LUCENE-7960
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: modules/analysis
>            Reporter: Shawn Heisey
>            Priority: Major
>         Attachments: LUCENE-7960.patch, LUCENE-7960.patch
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
> When ngram or edgengram filters are used, any terms that are shorter than the minGramSize
are completely removed from the token stream.
> This is probably 100% what was intended, but I've seen it cause a lot of problems for
users.  I am not suggesting that the default behavior be changed.  That would be far too disruptive
to the existing user base.
> I do think there should be a new boolean option, with a name like keepShortTerms, that
defaults to false, to allow the short terms to be preserved.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message