lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Noble Paul നോബിള്‍ नोब्ळ् <noble.p...@corp.aol.com>
Subject Re: ${dataimporter.delta.twitter_id} not getting populated in deltaImportQuery
Date Thu, 12 Nov 2009 05:54:30 GMT
are you sure the data comes back in the same name. Some DBs return the
field names in ALL CAPS

you may try out a delta_import using a full import too

http://wiki.apache.org/solr/DataImportHandlerFaq#My_delta-import_goes_out_of_memory_._Any_workaround_.3F

On Wed, Nov 11, 2009 at 9:55 PM, Mark Ellul <mark@catalystic.com> wrote:
> I have 2 entities from the root node, not sure if that makes a difference!
>
> On Wed, Nov 11, 2009 at 4:49 PM, Mark Ellul <mark@catalystic.com> wrote:
>
>> Hi,
>>
>> I have a interesting issue...
>>
>> Basically I am trying to delta imports on solr 1.4 on a postgresql 8.3
>> database.
>>
>> Basically when I am running a delta import with the entity below I get an
>> exception  (see below the entity definition) showing the query its trying to
>> run and you can see that its not populating the where clause of my
>> dataImportQuery.
>>
>> I have tried ${dataimporter.delta.twitter_id} and ${dataimporter.delta.id}
>> and get the same exceptions.
>>
>> Am I missing something obvious?
>>
>> Any help would be appreciated!
>>
>> Regards
>>
>> Mark
>>
>>
>> <entity name="Tweeter"  pk="twitter_id"
>>  query="
>> select twitter_id,
>> twitter_id as pk,
>>  1 as site_id,
>> screen_name
>>
>> from api_tweeter WHERE
>>  tweet_mapreduce_on IS NOT NULL;
>> " transformer="TemplateTransformer"
>>
>> deltaImportQuery="
>>                 select twitter_id,
>>  twitter_id as pk,
>> 1 as site_id,
>> screen_name
>>
>> from api_tweeter
>> where twitter_id=${dataimporter.delta.twitter_id };
>> "
>> deltaQuery ="select twitter_id from api_tweeter where  modified_on >
>> '${dataimporter.last_index_time}' and tweet_mapreduce_on IS NOT NULL;"
>>
>> >
>> <field name="twitter_id" column="twitter_id" />
>>
>>
>> </entity>
>>
>>
>> INFO: Completed parentDeltaQuery for Entity: Tweeter
>> Nov 11, 2009 3:35:44 PM org.apache.solr.handler.dataimport.DocBuilder
>> buildDocument
>> SEVERE: Exception while processing: Tweeter document :
>> SolrInputDocument[{}]
>> org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
>> execute query:                          select twitter_id,      
 twitter_id
>> as pk,        1 as site_id,       screen_name   from api_tweeter     where
>> twitter_id=;    Processing Document # 1
>>  at
>> org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
>> at
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:253)
>>  at
>> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210)
>> at
>> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
>>  at
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:58)
>> at
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71)
>>  at
>> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237)
>> at
>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:357)
>>  at
>> org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:276)
>> at
>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:172)
>>  at
>> org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:352)
>> at
>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:391)
>>  at
>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370)
>> Caused by: org.postgresql.util.PSQLException: ERROR: syntax error at end of
>> input
>>   Position: 1197
>> at
>> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2062)
>> at
>> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1795)
>>  at
>> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
>> at
>> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:479)
>>  at
>> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:353)
>> at
>> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:345)
>>  at
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:246)
>> ... 11 more
>> Nov 11, 2009 3:35:44 PM org.apache.solr.handler.dataimport.DataImporter
>> doDeltaImport
>> SEVERE: Delta Import Failed
>> org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
>> execute query:                          select twitter_id,      
 twitter_id
>> as pk,        1 as site_id,       screen_name  from api_tweeter     where
>> twitter_id=;    Processing Document # 1
>>  at
>> org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
>> at
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:253)
>>  at
>> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210)
>> at
>> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
>>  at
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:58)
>> at
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71)
>>  at
>> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237)
>> at
>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:357)
>>  at
>> org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:276)
>> at
>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:172)
>>  at
>> org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:352)
>> at
>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:391)
>>  at
>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370)
>> Caused by: org.postgresql.util.PSQLException: ERROR: syntax error at end of
>> input
>>   Position: 1197
>> at
>> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2062)
>> at
>> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1795)
>>  at
>> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
>> at
>> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:479)
>>  at
>> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:353)
>> at
>> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:345)
>>  at
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:246)
>> ... 11 more
>> Nov 11, 2009 3:35:44 PM org.apache.solr.update.DirectUpdateHandler2
>> rollback
>> INFO: start rollback
>> Nov 11, 2009 3:35:44 PM org.apache.solr.update.DirectUpdateHandler2
>> rollback
>> INFO: end_rollback
>>
>>
>



-- 
-----------------------------------------------------
Noble Paul | Principal Engineer| AOL | http://aol.com

Mime
View raw message