lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: svn commit: r1502468 - in /lucene/dev/trunk/solr/core/src: java/org/apache/solr/core/CorePropertiesLocator.java test/org/apache/solr/core/TestSolrXmlPersistor.java
Date Fri, 12 Jul 2013 08:53:25 GMT
Yes, and be sure to do the opposite when reading properties files! Otherwise it is not consistent
and loading/saving and exceptions may happen.

I am working on forbidden-apis to ensure we are consistent everywhere. The binary properties
file format is the officially defined one according to JVM spec.

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Alan Woodward [mailto:alan@flax.co.uk]
> Sent: Friday, July 12, 2013 10:47 AM
> To: dev@lucene.apache.org
> Subject: Re: svn commit: r1502468 - in /lucene/dev/trunk/solr/core/src:
> java/org/apache/solr/core/CorePropertiesLocator.java
> test/org/apache/solr/core/TestSolrXmlPersistor.java
> 
> Does this fix it?
> 
> @@ -79,9 +76,7 @@ public class CorePropertiesLocator implements
> CoresLocator {
>      OutputStream os = null;
>      try {
>        os = new FileOutputStream(propfile);
> -      Writer writer = new OutputStreamWriter(os, Charsets.UTF_8);
> -      p.store(writer, "Written by CorePropertiesLocator on " + new Date());
> -      writer.close();
> +      p.store(os, "Written by CorePropertiesLocator on " + new Date());
>      }
>      catch (IOException e) {
> 
> Alan Woodward
> www.flax.co.uk
> 
> 
> On 12 Jul 2013, at 09:35, Uwe Schindler wrote:
> 
> > Hi,
> >
> > you have tob e careful: If you store properties with a writer but load it with
> InputStream, the code is different. Properties files have a defined charset of
> ISO-8859-1:
> >
> > "The load(Reader) / store(Writer, String) methods load and store
> properties from and to a character based stream in a simple line-oriented
> format specified below. The load(InputStream) / store(OutputStream,
> String) methods work the same way as the load(Reader)/store(Writer,
> String) pair, except the input/output stream is encoded in ISO 8859-1
> character encoding. Characters that cannot be directly represented in this
> encoding can be written using Unicode escapes as defined in section 3.3 of
> The Java™ Language Specification; only a single 'u' character is allowed in an
> escape sequence. The native2ascii tool can be used to convert property files
> to and from other character encodings."
> >
> > So be sure to be consistent when loading/saving! If we previously (in older
> Solr version) used the InputStream methods to load/store core props, we
> should use ISO-8859-1 to load/store to be compatible with older versions!
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: uwe@thetaphi.de
> >
> >
> >> -----Original Message-----
> >> From: romseygeek@apache.org [mailto:romseygeek@apache.org]
> >> Sent: Friday, July 12, 2013 10:26 AM
> >> To: commits@lucene.apache.org
> >> Subject: svn commit: r1502468 - in /lucene/dev/trunk/solr/core/src:
> >> java/org/apache/solr/core/CorePropertiesLocator.java
> >> test/org/apache/solr/core/TestSolrXmlPersistor.java
> >>
> >> Author: romseygeek
> >> Date: Fri Jul 12 08:25:36 2013
> >> New Revision: 1502468
> >>
> >> URL: http://svn.apache.org/r1502468
> >> Log:
> >> SOLR-4914: Close OutputStreamWriter properly, use
> >> System.getProperty("line.separator") instead of \n
> >>
> >> Fixes Windows test failures.
> >>
> >> Modified:
> >>
> >> lucene/dev/trunk/solr/core/src/java/org/apache/solr/core/CoreProperti
> >> esL
> >> ocator.java
> >>
> >> lucene/dev/trunk/solr/core/src/test/org/apache/solr/core/TestSolrXmlP
> >> ersi
> >> stor.java
> >>
> >> Modified:
> >> lucene/dev/trunk/solr/core/src/java/org/apache/solr/core/CoreProperti
> >> esL
> >> ocator.java
> >> URL:
> >> http://svn.apache.org/viewvc/lucene/dev/trunk/solr/core/src/java/org/
> >> apa
> >>
> che/solr/core/CorePropertiesLocator.java?rev=1502468&r1=1502467&r2=15
> >> 0
> >> 2468&view=diff
> >>
> ==========================================================
> >> ====================
> >> ---
> >> lucene/dev/trunk/solr/core/src/java/org/apache/solr/core/CoreProperti
> >> esL
> >> ocator.java (original)
> >> +++
> lucene/dev/trunk/solr/core/src/java/org/apache/solr/core/CoreProp
> >> +++ ert iesLocator.java Fri Jul 12 08:25:36 2013
> >> @@ -20,6 +20,7 @@ package org.apache.solr.core;  import
> >> com.google.common.base.Charsets;  import
> >> com.google.common.collect.Lists;  import
> >> org.apache.solr.common.SolrException;
> >> +import org.apache.solr.util.IOUtils;
> >> import org.slf4j.Logger;
> >> import org.slf4j.LoggerFactory;
> >>
> >> @@ -27,6 +28,7 @@ import java.io.File; import
> >> java.io.FileInputStream; import java.io.FileOutputStream; import
> >> java.io.IOException;
> >> +import java.io.OutputStream;
> >> import java.io.OutputStreamWriter;
> >> import java.io.Writer;
> >> import java.util.Date;
> >> @@ -56,14 +58,7 @@ public class CorePropertiesLocator imple
> >>         throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> >>                                 "Could not create a new core in " + cd.getInstanceDir()
> >>                               + "as another core is already defined there");
> >> -      try {
> >> -        Properties p = buildCoreProperties(cd);
> >> -        Writer writer = new OutputStreamWriter(new
> >> FileOutputStream(propFile), Charsets.UTF_8);
> >> -        p.store(writer, "Written by CorePropertiesLocator on " + new
> Date());
> >> -      }
> >> -      catch (IOException e) {
> >> -        logger.error("Couldn't persist core properties to {}: {}",
> >> propFile.getAbsolutePath(), e);
> >> -      }
> >> +      writePropertiesFile(cd, propFile);
> >>     }
> >>   }
> >>
> >> @@ -75,14 +70,25 @@ public class CorePropertiesLocator imple
> >>   public void persist(CoreContainer cc, CoreDescriptor... coreDescriptors) {
> >>     for (CoreDescriptor cd : coreDescriptors) {
> >>       File propFile = new File(new File(cd.getInstanceDir()),
> >> PROPERTIES_FILENAME);
> >> -      try {
> >> -        Properties p = buildCoreProperties(cd);
> >> -        Writer writer = new OutputStreamWriter(new
> >> FileOutputStream(propFile), Charsets.UTF_8);
> >> -        p.store(writer, "Written by CorePropertiesLocator on " + new
> Date());
> >> -      }
> >> -      catch (IOException e) {
> >> -        logger.error("Couldn't persist core properties to {}: {}",
> >> propFile.getAbsolutePath(), e);
> >> -      }
> >> +      writePropertiesFile(cd, propFile);
> >> +    }
> >> +  }
> >> +
> >> +  private void writePropertiesFile(CoreDescriptor cd, File propfile)  {
> >> +    Properties p = buildCoreProperties(cd);
> >> +    OutputStream os = null;
> >> +    try {
> >> +      os = new FileOutputStream(propfile);
> >> +      Writer writer = new OutputStreamWriter(os, Charsets.UTF_8);
> >> +      p.store(writer, "Written by CorePropertiesLocator on " + new Date());
> >> +      writer.close();
> >> +    }
> >> +    catch (IOException e) {
> >> +      logger.error("Couldn't persist core properties to {}: {}",
> >> propfile.getAbsolutePath(), e);
> >> +    }
> >> +    finally {
> >> +      if (os != null)
> >> +        IOUtils.closeQuietly(os);
> >>     }
> >>   }
> >>
> >>
> >> Modified:
> >> lucene/dev/trunk/solr/core/src/test/org/apache/solr/core/TestSolrXmlP
> >> ersi
> >> stor.java
> >> URL:
> >> http://svn.apache.org/viewvc/lucene/dev/trunk/solr/core/src/test/org/
> >> apa
> >>
> che/solr/core/TestSolrXmlPersistor.java?rev=1502468&r1=1502467&r2=150
> >> 2
> >> 468&view=diff
> >>
> ==========================================================
> >> ====================
> >> ---
> >> lucene/dev/trunk/solr/core/src/test/org/apache/solr/core/TestSolrXmlP
> >> ersi
> >> stor.java (original)
> >> +++ lucene/dev/trunk/solr/core/src/test/org/apache/solr/core/TestSolr
> >> +++ Xml Persistor.java Fri Jul 12 08:25:36 2013
> >> @@ -70,8 +70,8 @@ public class TestSolrXmlPersistor {
> >>
> >>     SolrXMLCoresLocator persistor = new SolrXMLCoresLocator(new
> >> File("testfile.xml"), solrxml, null);
> >>     assertEquals(persistor.buildSolrXML(cds),
> >> -          "<solr><cores>\n"
> >> -        + "    <core name=\"testcore\" instanceDir=\"instance/dir/\"/>\n"
> >> +          "<solr><cores>" + SolrXMLCoresLocator.NEWLINE
> >> +        + "    <core name=\"testcore\" instanceDir=\"instance/dir/\"/>"
+
> >> SolrXMLCoresLocator.NEWLINE
> >>         + "</cores></solr>");
> >>   }
> >>
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> > additional commands, e-mail: dev-help@lucene.apache.org
> >
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message