lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "david.w.smiley@gmail.com" <david.w.smi...@gmail.com>
Subject Re: Indexing and Querying MS SQL Server 2012 Spatial
Date Sun, 24 Aug 2014 03:22:20 GMT
What’s with the double square brackets around your WKT?  Remove them.
~ David Smiley
Freelance Apache Lucene/Solr Search Consultant/Developer
http://www.linkedin.com/in/davidwsmiley


On Sat, Aug 23, 2014 at 10:37 PM, Bostic, Alex <alex.bostic@urs.com> wrote:
> Hello, now that I have the WKT in a sting field, I get this during the dataimport process
>
> org.apache.solr.common.SolrException: Couldn't parse shape '[[POLYGON ((-8134690.8140452942
4866553.78506005, -7836280.655620046 4884898.671848488, -8009945.58388392 4732024.615278176,
-8134690.8140452942 4866553.78506005))]]]' because: For input string: "[[POLYGON ((-8134690.8140452942
4866553.78506005"
>         at org.apache.solr.schema.AbstractSpatialFieldType.parseShape(AbstractSpatialFieldType.java:175)
>         at org.apache.solr.schema.AbstractSpatialFieldType.createFields(AbstractSpatialFieldType.java:139)
>         at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:47)
>         at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:118)
>         at org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:77)
>         at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:238)
>         at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:164)
>         at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
>         at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
>         at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:870)
>         at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1024)
>         at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:693)
>         at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
>         at org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:71)
>         at org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:261)
>         at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:496)
>         at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:400)
>         at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:315)
>         at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:223)
>         at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:411)
>         at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:483)
>         at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:464)
> Caused by: java.lang.NumberFormatException: For input string: "[[POLYGON ((-8134690.8140452942
4866553.78506005"
>         at sun.misc.FloatingDecimal.readJavaFormatString(Unknown Source)
>         at java.lang.Double.parseDouble(Unknown Source)
>         at com.spatial4j.core.io.ParseUtils.parsePointDouble(ParseUtils.java:108)
>         at com.spatial4j.core.io.ParseUtils.parseLatitudeLongitude(ParseUtils.java:145)
>         at com.spatial4j.core.io.ParseUtils.parseLatitudeLongitude(ParseUtils.java:137)
>         at com.spatial4j.core.io.LegacyShapeReadWriterFormat.readLatCommaLonPoint(LegacyShapeReadWriterFormat.java:169)
>         at com.spatial4j.core.io.LegacyShapeReadWriterFormat.readShapeOrNull(LegacyShapeReadWriterFormat.java:153)
>         at org.apache.solr.schema.AbstractSpatialFieldType.parseShape(AbstractSpatialFieldType.java:167)
>
> my field type is defined as follows
> <fieldType name="location_rpt" class="solr.SpatialRecursivePrefixTreeFieldType" spatialContextFactory="com.spatial4j.core.context.jts.JtsSpatialContextFactory"
>         geo="false" distErrPct="0.025" maxDistErr="0.000009" units="degrees" />
>
> Alex Bostic
> GIS Developer
> URS Corporation
> 12420 Milestone Center Drive, Suite 150
> Germantown, MD 20876
> direct line: 301-820-3287
> cell line: 301-213-2639
>
>
> -----Original Message-----
> From: Bostic, Alex [mailto:alex.bostic@urs.com]
> Sent: Saturday, August 23, 2014 2:27 PM
> To: solr-user@lucene.apache.org
> Subject: RE: Indexing and Querying MS SQL Server 2012 Spatial
>
> Thanks I will try for a correct install. I have many other fields that are indexing fine,
 I will inspect for further errors and use DIH debug mode. I will also create a string field
for geometries.
>
> One item to note is that the spatial data is stored in sql servers geometry type field.
Only through the DIH query do I return WKT, maybe that's the issue.
> ________________________________________
> From: david.w.smiley@gmail.com [david.w.smiley@gmail.com]
> Sent: Saturday, August 23, 2014 2:03 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Indexing and Querying MS SQL Server 2012 Spatial
>
> Putting JTS where you put it is fine temporarily but I strongly recommend against any
jar being placed there at all; ideally it should be associated with the Solr installation;
not with java itself.
>
> Some kinda of errors must be happening but you're not seeing them.  I presume you've
configured the DIH since you're telling me your hitting its button in the UI.  The DIH has
a debug mode; it's been a while since I used the DIH last but I remember that feature being
really useful.
>
> For the sake of troubleshooting, try putting the geometries (WKT) into a "string" field,
then you should see them.  If not then you're current problem is squarely with the import
process/config, not with Solr spatial.
> ~ David Smiley
> Freelance Apache Lucene/Solr Search Consultant/Developer http://www.linkedin.com/in/davidwsmiley
>
>
> On Sat, Aug 23, 2014 at 10:53 AM, Bostic, Alex <alex.bostic@urs.com> wrote:
>> Ok thanks, I am even closer
>>
>> I put jts-1.13.jar here C:\Program Files (x86)\Java\jre7\lib\ext
>>
>> And removed it from the cmd line when starting solr
>>
>> I no longer get the class not found issues.  I removed the jar from the above location
and got the error again, so I put it back to confirm I'm past the Not Found issue.
>>
>> My core appears to startup fine,
>>
>> I select my core in the browser manager and Execute a full-import with
>> Clean and Commit checked.  That appears to go well.  However, I still
>> don't see my geom field as part of my index
>>
>> I do ALSO have the jar in
>> C:\AddSoftware\solr-4.9.0\ocsirasspatial\solr-webapp\webapp\WEB-INF\li
>> b
>>
>> Any other hints are certainly welcome.  I think I'm close
>>
>> Alex Bostic
>> GIS Developer
>> URS Corporation
>> 12420 Milestone Center Drive, Suite 150 Germantown, MD 20876 direct
>> line: 301-820-3287 cell line: 301-213-2639
>>
>>
>> -----Original Message-----
>> From: david.w.smiley@gmail.com [mailto:david.w.smiley@gmail.com]
>> Sent: Saturday, August 23, 2014 9:15 AM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Indexing and Querying MS SQL Server 2012 Spatial
>>
>> The jts-1.13.jar file (other JTS jars that come with it aren't used) needs to go
on the java classpath.  I haven't tried to do it using the way you're doing it.  Note that
the "lib" directories Solr manages won't work for some jar's like this one.  I forget but
I think putting it in examples/lib works.  Unpacking the WAR and putting it in WEB-INF/lib
definitely works.
>> ~ David Smiley
>> Freelance Apache Lucene/Solr Search Consultant/Developer
>> http://www.linkedin.com/in/davidwsmiley
>>
>>
>> On Sat, Aug 23, 2014 at 4:36 AM, Bostic, Alex <alex.bostic@urs.com> wrote:
>>> Ok, I was missing
>>> spatialContextFactory="com.spatial4j.core.context.jts.JtsSpatialContextFactory"
>>>
>>>
>>> Now I am getting
>>>
>>> Error loading core:java.util.concurrent.ExecutionException:
>>> java.lang.NoClassDefFoundError:
>>> com/vividsolutions/jts/geom/CoordinateSequenceFactory
>>>
>>> Any help resolving my path issues would be great
>>>
>>> Alex Bostic
>>> GIS Developer
>>> URS Corporation
>>> 12420 Milestone Center Drive, Suite 150 Germantown, MD 20876 direct
>>> line: 301-820-3287 cell line: 301-213-2639
>>>
>>>
>>> -----Original Message-----
>>> From: Bostic, Alex [mailto:alex.bostic@urs.com]
>>> Sent: Saturday, August 23, 2014 3:53 AM
>>> To: solr-user@lucene.apache.org
>>> Subject: RE: Indexing and Querying MS SQL Server 2012 Spatial
>>>
>>> Hello, I guess I'm closer. But my geom field does not show up in the index.
>>>
>>>   I have a table full of polys, stored as SQL Server Geometry Type.
>>>
>>> Taken from the example schema I have:
>>> ..
>>> <field name="geom" type="location_rpt" indexed="true" stored="true"/> ..
>>>
>>> In my data-config.xml I have:
>>> <entity name="data" dataSource="ds1" pk="key"
>>>     query="SELECT wt.Id, wt.Name, wt.WorkTypeTypeId,wt.ChildId,wt.CreatedDate,wt.ModifiedDate,wt.Year,
wta.GEOM.STAsText(), wta.OBJECTID
>>>   FROM dbo.WorkTypes wt Inner Join dbo.WorkTypeAreas wta ON wt.Id = wta.WorkTypeId">
>>>       <field column="Id" name="id" />
>>>       <field column="ChildId" name="childid" />
>>>       <field column="WorkTypeTypeId" name="worktypetypeid" />
>>>       <field column="Name" name="name" />
>>>       <field column="CreatedDate" name="createddate" />
>>>           <field column="ModifiedDate" name="modifieddate" />
>>>           <field column="UserId" name="userid" />
>>>           <field column="Year" name="year" />
>>>           <field column="GEOM" name="geom" />
>>>           <field column="OBJECTID" name="objectid" />
>>>     </entity>
>>>
>>> OBJECTID shows up fine in my updated index, gut GEOM does not.  The
>>> index was successful and I don't see any errors in the console or the
>>> log (from the web)
>>>
>>> I am starting solr like so:
>>> java -Dsolr.solr.home=c:\AddedSoftware\solr-4.90\ocsirasspatial\solr
>>> -Djetty.class.path=c:\AddedSoftware\solr-4.9.0\jts-1.13\lib -jar
>>> start.jar
>>>
>>> Based on the above, what am I missing to get this to work.  Maybe I am overlooking
an issue in the console?
>>>
>>> Thanks
>>>
>>>
>>> Alex Bostic
>>> GIS Developer
>>> URS Corporation
>>> 12420 Milestone Center Drive, Suite 150 Germantown, MD 20876 direct
>>> line: 301-820-3287 cell line: 301-213-2639
>>>
>>>
>>> -----Original Message-----
>>> From: Bostic, Alex [mailto:alex.bostic@urs.com]
>>> Sent: Wednesday, August 20, 2014 4:41 PM
>>> To: solr-user@lucene.apache.org
>>> Subject: RE: Indexing and Querying MS SQL Server 2012 Spatial
>>>
>>> Ok Great, I'm just going to dive in and see if I can index my data.  Does spatial
reference matter?
>>>
>>> Alex Bostic
>>> GIS Developer
>>> URS Corporation
>>> 12420 Milestone Center Drive, Suite 150 Germantown, MD 20876 direct
>>> line: 301-820-3287 cell line: 301-213-2639
>>>
>>>
>>> -----Original Message-----
>>> From: Pires, Guilherme [mailto:guilherme.pires@cgi.com]
>>> Sent: Wednesday, August 20, 2014 4:30 PM
>>> To: solr-user@lucene.apache.org
>>> Subject: RE: Indexing and Querying MS SQL Server 2012 Spatial
>>>
>>> Hello,
>>>
>>> I've been working with Solr together with JTS and use location_rpt
>>> class for the geometry field for a while now. (However, I must say
>>> that the index grew a lot when used this class instead of the geohash
>>> for simple points ..so use it only if you really need to index
>>> polylines and/or polygons)
>>>
>>> I actually already successfully connected solr to postGis and oracle spatial
via DIH but in this live website ( http://cascaismap.com ) we had a GE Smallworld as the GIS
system so it was easier just to build a sync engine that periodically queries differences
from the GIS and push them into solr via xml document. This project has already couple of
years now so a lot would be different now.
>>>
>>> In that website, solr provides, obviously, all the text search on the top and
also 70% of the themes available on the treeview on the left (expand in the red button) that
are result of a bounding box query to geometry index in solr.
>>> Something like this : (...)q=bounds:Intersects(-9.463118366688718
>>> 38.67913579372146 -9.370549969166746 38.7109390712568)(...)
>>>
>>> After this, we actually provided for a different project, a similar sync mechanism
but between in-house solr instances and google maps engine datastore in the cloud and it works
like a charm.
>>>
>>> Guilherme Pires
>>> Geospatial Intelligence @ CGI
>>> guilherme.pires@cgi.com
>>>
>>> ________________________________________
>>> De: david.w.smiley@gmail.com [david.w.smiley@gmail.com]
>>> Enviado: quarta-feira, 20 de Agosto de 2014 18:49
>>> Para: solr-user@lucene.apache.org
>>> Assunto: Re: Indexing and Querying MS SQL Server 2012 Spatial
>>>
>>> Hi Alex,
>>>
>>> I guess a spatial tutorial might be helpful, but there isn't one.  There is a
sample at the Lucene-spatial layer but not up at Solr.  You need to use WKT syntax for line's
and polys, and you may do so as well for other shapes.  And in the schema use location_rpt
copied from Solr's example schema for starters, but modified as the ref guide & wiki show
to use JTS.
>>>  The ref guide, wiki, and I would guess that book should show how to to a bounding
box query using {!bbox} - it's pretty simple.
>>>
>>> ~ David Smiley
>>> Freelance Apache Lucene/Solr Search Consultant/Developer
>>> http://www.linkedin.com/in/davidwsmiley
>>>
>>>
>>> On Tue, Aug 19, 2014 at 11:25 AM, Bostic, Alex <alex.bostic@urs.com> wrote:
>>>
>>>> Hello I'm new to Solr:
>>>> I have a SQL Server 2012 database with spatial columns
>>>> (points/lines/polys) Do you have any resources to point to for the
>>>> following Creating a Solr index of a sql server spatial table
>>>> Bounding Box query (intersect) example, possibly with front-end from
>>>> GMaps or OpenLayers  I'm currently reading Apache Solr Beginner's
>>>> Guide and have reviewed
>>>> https://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4
>>>> I am able to index and query my non spatial data, I am just looking
>>>> for some resource that may have some more detail about how to set everything
up.
>>>> I can provide more detail if needed.
>>>> Thanks
>>>>
>>>> Alex Bostic
>>>> GIS Developer
>>>> URS Corporation
>>>> 12420 Milestone Center Drive, Suite 150 Germantown, MD 20876 direct
>>>> line: 301-820-3287 cell line: 301-213-2639
>>>>
>>>>
>>>>
>>>> This e-mail and any attachments contain URS Corporation confidential
>>>> information that may be proprietary or privileged. If you receive
>>>> this message in error or are not the intended recipient, you should
>>>> not retain, distribute, disclose or use any of this information and
>>>> you should destroy the e-mail and any attachments or copies.
>>>>
>>>
>>>
>>> This e-mail and any attachments contain URS Corporation confidential information
that may be proprietary or privileged. If you receive this message in error or are not the
intended recipient, you should not retain, distribute, disclose or use any of this information
and you should destroy the e-mail and any attachments or copies.

Mime
View raw message