lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Khludnev <m...@apache.org>
Subject Re: DataImportHandler OutOfMemory Mysql
Date Sun, 02 Apr 2017 06:03:57 GMT
Hello, Sergio.

Have you tried Integer.MIN_VALUE ? -2147483648 see
https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-implementation-notes.html


On Sun, Apr 2, 2017 at 1:17 AM, marotosg <marotosg@gmail.com> wrote:

> Hi,
>
> I am trying to load a big table into Solr using DataImportHandler and
> Mysql.
> I am getting OutOfMemory error because Solr is trying to load the full
> table. I have been reading different posts and tried batchSize="-1".
> https://wiki.apache.org/solr/DataImportHandlerFaq
>
> Do you have any idea what could be the issue?
> Completely lost here.
>
> Solr.6.4.1
> mysql-connector-java-5.1.41-bin.jar
>
> data-config
>
> <dataSource type="JdbcDataSource"
>             driver="com.mysql.jdbc.Driver"
>             url="jdbc:mysql://188.68.190.85:3306/jobsdb"
>             user="suer"
>             password="passowrd"/>
> <document>
>   <entity name="jobsearch"
>     pk="id"
>         batchSize="-1"
>     query="select * from job"
>     deltaImportQuery="SELECT * from job WHERE id='${dih.delta.id}'"
>     deltaQuery="SELECT id FROM job  WHERE updated_at >
> '${dih.last_index_time}'"
>     >
>      <field column="job_id" name="JobID"/>
>      <field column="position" name="Position"/>
>      <field column="employment_type" name="EmploymentType"/>
>          <field column="description" name="Description"/>
>      <field column="category" name="Category"/>
>          <field column="apply_url" name="ApplyUrl"/>
>      <field column="description_url" name="DescriptionUrl"/>
>          <field column="company" name="Company"/>
>          <field column="city" name="City"/>
>          <field column="country_subdivision1" name="CountrySubdivision1"/>
>          <field column="country_subdivision2" name="CountrySubdivision2"/>
>          <field column="country" name="Country"/>
>      <field column="source" name="Source"/>
>          <field column="created_at" name="CreatedAt"/>
>          <field column="updated_at" name="UpdatedAt"/>
>   </entity>
> </document>
> </dataConfig>
>
> Thanks
> Sergio
>
>
>
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/DataImportHandler-OutOfMemory-Mysql-tp4327982.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



-- 
Sincerely yours
Mikhail Khludnev

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message