lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: StandardTokenizerImpl generation question
Date Sat, 25 Apr 2015 09:13:38 GMT
Hi,

I checked the source codes of Lucene, there is no such warning anymore. Which version do you
use and in which file did you find the warning? In any case, for older versions of Lucene
you should really use the version mentioned there to regenerate, to be sure the behavior of
the Tokenizer is the one you want to have.

It is easy to do if you use the corresponding ANT task for regenerating ("ant jflex" or "ant
regenerate").

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Uwe Schindler [mailto:uwe@thetaphi.de]
> Sent: Saturday, April 25, 2015 10:34 AM
> To: java-user@lucene.apache.org
> Subject: RE: StandardTokenizerImpl generation question
> 
> Hi,
> 
> the documentation is this file is not really up-to-date.
> 
> If you want to regenerate the jflex files, it is recommended to use the build
> script of Lucene. If you type "ant regenerate" or "ant jflex" (in the folder of
> the analysis module), the build system will download the correct version of
> JFlex and regenerate the scripts. Currently this is version 1.6.0. The warning
> was primarily there because of older versions of JFlex that used the Unicode
> version of  the JVM which is no longer the case.
> 
> Could you may be open an issue to update the documentation in the files?
> 
> Uwe
> 
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
> 
> > -----Original Message-----
> > From: Youngho Cho [mailto:youngho1203@gmail.com]
> > Sent: Saturday, April 25, 2015 4:40 AM
> > To: java-user@lucene.apache.org
> > Subject: StandardTokenizerImpl generation question
> >
> > Hello,
> >
> > I would like to generate my own tokenizer which is slightly modify the
> > StandardTokenizerImpl*.jflex.
> >
> > But I read the READ_BEFORE_GENERATING.txt.
> >
> > Currently, the jflex version is 1.6.1. but the document recommend to
> > use jflex 1.5 only.
> >
> > Still this message is valid and do I have to generate a tokenizer
> > using old version of jflex ?
> >
> >
> > Thanks,
> >
> > Youngho
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message