lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Perry <ju...@limitless.co.uk>
Subject Re: Build index from Oracle, adding fields
Date Fri, 27 Mar 2015 00:11:32 GMT
On 27/03/2015 12:42, Shawn Heisey wrote:
> If that's not practical, then the only real option you have is to drop
> back to one entity, and build a single SELECT statement (using JOIN and
> some form of CONCAT) that will gather all the information from all the
> tables at the same time, and combine multiple values together into one
> SQL result field with some kind of delimiter.  Then you can use the
> RegexTransformer's "splitBy" functionality to turn the concatenated data
> back into multiple values for your multi-valued field.  Database servers
> tend to be REALLY good at JOIN operations, so the database would be
> doing the heavy lifting.

I did try that in fact (and do it with one of my other indexes).

However, with this index the sub-select can return 200 rows of
200 characters - and that blows up in Oracle as the field is
over 4000 characters long (and the work-around for that is to
use clob's - but that has its own performance problems).

Currently I am doing this by exporting a CSV file and
processing it with a C program - and then reading the CSV with
SOLR :(

-- 
Cheers
Jules.


Mime
View raw message