lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ashimbose <ashimb...@gmail.com>
Subject Large data importing getting rollback with solr
Date Thu, 17 Jan 2013 07:43:14 GMT
I am trying to index large data (not rich document) about 5GB, but Its not
getting index. In case of small data it's perfectly indexing.For Large data
import XML response..      0    0              data-config.xml       
full-import  busy  A command is still running...      0:9:12.738    169   
181079    0    2013-01-17 12:50:13    Indexing failed. Rolled back all
changes.    2013-01-17 12:50:30    This response format is experimental.  It
is likely to change in the future.BUT for small data index XML response
perfectly OK as below...      0    0              data-config.xml       
full-import  busy  A command is still running...      0:0:12.436    11   
38209    0    2013-01-17 12:56:57    Indexing completed. Added/Updated:
38209 documents. Deleted 0 documents.    This response format is
experimental.  It is likely to change in the future.For Large data error log
response is as below...Its getting RollbackINFO: Time taken for
getConnection(): 1343Jan 17, 2013 12:36:21 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOBCODE_HAZ_BRA with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:36:23 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
callINFO: Time taken for getConnection(): 1341Jan 17, 2013 12:36:23 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOBCODE_HAZ_TBL with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:36:24 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
callINFO: Time taken for getConnection(): 1357Jan 17, 2013 12:36:24 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOBCODE_LANG with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:36:26 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
callINFO: Time taken for getConnection(): 1392Jan 17, 2013 12:36:26 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOBCODE_TBL with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:36:27 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
callINFO: Time taken for getConnection(): 1535Jan 17, 2013 12:36:41 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOBCODE_TBL_ARG with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:36:43 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
callINFO: Time taken for getConnection(): 1467Jan 17, 2013 12:36:43 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOBCODE_TBL_BRA with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:36:44 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
callINFO: Time taken for getConnection(): 1373Jan 17, 2013 12:36:44 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOBCOMP_TMP_MC with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:36:45 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
callINFO: Time taken for getConnection(): 1404Jan 17, 2013 12:36:45 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOBFUNCTION_LNG with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:36:47 PM org.apache.solr.core.SolrCore executeINFO: [core1]
webapp=/solr path=/dataimport params={command=full-import} status=0
QTime=0Jan 17, 2013 12:36:47 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Time taken for
getConnection(): 1357Jan 17, 2013 12:36:47 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOBFUNCTION_TBL with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:36:48 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
callINFO: Time taken for getConnection(): 1310Jan 17, 2013 12:36:48 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOB_APPROVALS with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:36:50 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
callINFO: Time taken for getConnection(): 1342Jan 17, 2013 12:36:50 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOB_AUS with URL:
jdbc:attconnect://ESMART12:2551/NAVIGATOR;DefTdpName=sampleDBJan 17, 2013
12:36:53 PM org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO:
Time taken for getConnection(): 2979Jan 17, 2013 12:36:54 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOB_CTG_FRA_LNG with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:36:56 PM org.apache.solr.core.SolrCore executeINFO: [core1]
webapp=/solr path=/dataimport params={command=full-import} status=0
QTime=0Jan 17, 2013 12:36:56 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Time taken for
getConnection(): 2231Jan 17, 2013 12:36:56 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOB_CTG_FRA_TBL with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:36:58 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
callINFO: Time taken for getConnection(): 1322Jan 17, 2013 12:36:58 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOB_EARNS_DIST with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:36:58 PM org.apache.solr.core.SolrCore executeINFO: [core1]
webapp=/solr path=/dataimport params={command=full-import} status=0
QTime=1Jan 17, 2013 12:36:59 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Time taken for
getConnection(): 1262Jan 17, 2013 12:36:59 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOB_ERN_DST_TMP with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:37:00 PM org.apache.solr.core.SolrCore executeINFO: [core1]
webapp=/solr path=/dataimport params={command=full-import} status=0
QTime=0Jan 17, 2013 12:37:01 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Time taken for
getConnection(): 1272Jan 17, 2013 12:37:01 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOB_FAMILY_LNG with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:37:01 PM org.apache.solr.core.SolrCore executeINFO: [core1]
webapp=/solr path=/dataimport params={command=full-import} status=0
QTime=0Jan 17, 2013 12:37:02 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Time taken for
getConnection(): 1373Jan 17, 2013 12:37:02 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOB_FAMILY_TBL with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:37:02 PM org.apache.solr.core.SolrCore executeINFO: [core1]
webapp=/solr path=/dataimport params={command=full-import} status=0
QTime=0Jan 17, 2013 12:37:03 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Time taken for
getConnection(): 1554Jan 17, 2013 12:37:04 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity PS_JOB_FNC_SUB_TBL with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:37:04 PM org.apache.solr.core.SolrCore executeINFO: [core1]
webapp=/solr path=/dataimport params={command=full-import} status=0
QTime=0Jan 17, 2013 12:37:05 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Time taken for
getConnection(): 1347Jan 17, 2013 12:37:05 PM
org.apache.solr.handler.dataimport.JdbcDataSource$1 callINFO: Creating a
connection for entity JOB_IND with URL:
jdbc:attconnect://192.168.1.29:2551/NAVIGATOR;DefTdpName=sampleDBJan 17,
2013 12:37:05 PM org.apache.solr.core.SolrCore executeINFO: [core1]
webapp=/solr path=/dataimport params={command=full-import} status=0
QTime=0Jan 17, 2013 12:37:07 PM org.apache.solr.common.SolrException
logSEVERE: Exception while processing: JOB_IND document :
SolrInputDocument[{}]:org.apache.solr.handler.dataimport.DataImportHandlerException:
Unable to execute query: select * FROM sampleDB:JOB_IND Processing Document
# 181054        at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
      
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:253)
      
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210)       
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)       
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
      
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
      
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.pullRow(EntityProcessorWrapper.java:330)
      
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:296)
      
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:683)       
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:619)       
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:327)       
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:225)       
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:375)      

at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:445)       
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:426)Caused
by: java.sql.SQLException: JBC0016E: Remote call failed (return
code=-2,220). SDK9019E: internal error        SDK9019X:        at
com.ibm.optim.connect.jdbc.NvSQLWarning.createSQLException(Unknown Source)       
at com.ibm.optim.connect.comm.Server.checkCallRetOld(Unknown Source)       
at com.ibm.optim.connect.navapi.SysTdp.bindTdpAndDestroy(Unknown Source)       
at com.ibm.optim.connect.jdbc.NvConnectionBase.open(Unknown Source)       
at com.ibm.optim.connect.jdbc.NvConnectPropertiesBase.openConnection(Unknown
Source)        at
com.ibm.optim.connect.jdbc.NvDriverCoreBase.connect(Unknown Source)       
at com.ibm.optim.connect.jdbc.NvDriverCoreBase.connect(Unknown Source)       
at
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:161)       
at
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:129)       
at
org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:363) 
     
at
org.apache.solr.handler.dataimport.JdbcDataSource.access$200(JdbcDataSource.java:39)     
 
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:240)
      
... 13 moreJan 17, 2013 12:37:07 PM
org.apache.solr.update.processor.LogUpdateProcessor finishINFO:
{deleteByQuery=*:*,add=[(null), (null), (null), (null), (null), (null),
(null), (null), ... (181053 adds)]} 0 17Jan 17, 2013 12:37:07 PM
org.apache.solr.common.SolrException logSEVERE: Full Import
failed:java.lang.RuntimeException: java.lang.RuntimeException:
org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
execute query: select * FROM sampleDB:JOB_IND Processing Document # 181054       
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:264)       
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:375)      

at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:445)       
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:426)Caused
by: java.lang.RuntimeException:
org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
execute query: select * FROM sampleDB:JOB_IND Processing Document # 181054       
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:621)       
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:327)       
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:225)       
... 3 moreCaused by:
org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
execute query: select * FROM sampleDB:JOB_IND Processing Document # 181054       
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
      
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:253)
      
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210)       
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)       
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
      
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
      
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.pullRow(EntityProcessorWrapper.java:330)
      
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:296)
      
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:683)       
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:619)       
... 5 moreCaused by: java.sql.SQLException: JBC0016E: Remote call failed
(return code=-2,220). SDK9019E: internal error        SDK9019X:        at
com.ibm.optim.connect.jdbc.NvSQLWarning.createSQLException(Unknown Source)       
at com.ibm.optim.connect.comm.Server.checkCallRetOld(Unknown Source)       
at com.ibm.optim.connect.navapi.SysTdp.bindTdpAndDestroy(Unknown Source)       
at com.ibm.optim.connect.jdbc.NvConnectionBase.open(Unknown Source)       
at com.ibm.optim.connect.jdbc.NvConnectPropertiesBase.openConnection(Unknown
Source)        at
com.ibm.optim.connect.jdbc.NvDriverCoreBase.connect(Unknown Source)       
at com.ibm.optim.connect.jdbc.NvDriverCoreBase.connect(Unknown Source)       
at
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:161)       
at
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:129)       
at
org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:363) 
     
at
org.apache.solr.handler.dataimport.JdbcDataSource.access$200(JdbcDataSource.java:39)     
 
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:240)
      
... 13 moreJan 17, 2013 12:37:07 PM
org.apache.solr.update.DirectUpdateHandler2 rollbackINFO: start rollbackJan
17, 2013 12:37:07 PM org.apache.solr.core.SolrCore executeINFO: [core1]
webapp=/solr path=/dataimport params={command=full-import} status=0
QTime=0Jan 17, 2013 12:37:08 PM org.apache.solr.update.DirectUpdateHandler2
rollbackINFO: end_rollbackJan 17, 2013 12:37:08 PM
org.apache.solr.core.SolrCore executeINFO: [core1] webapp=/solr
path=/dataimport params={command=full-import} status=0 QTime=0Please Please
help me, how to do large data indexing. 



--
View this message in context: http://lucene.472066.n3.nabble.com/Large-data-importing-getting-rollback-with-solr-tp4034075.html
Sent from the Solr - User mailing list archive at Nabble.com.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message