lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Khludnev (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SOLR-4799) SQLEntityProcessor for zipper join
Date Mon, 01 Dec 2014 14:09:13 GMT

     [ https://issues.apache.org/jira/browse/SOLR-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mikhail Khludnev updated SOLR-4799:
-----------------------------------
    Attachment: SOLR-4799.patch

yep. agree. Zipper is made optional and baked by factory method. 

I either didn't get your second point or I'd like to address this refactoring separately.
It seems like DIHCacheSupport, Zipper, and straightforward case with N queries (one per every
parent row) should be covered under separate abstraction of rowIterator, which will be reset
per every parent row. nevertheless it's more than moderate long story. Isn't it? 

> SQLEntityProcessor for zipper join
> ----------------------------------
>
>                 Key: SOLR-4799
>                 URL: https://issues.apache.org/jira/browse/SOLR-4799
>             Project: Solr
>          Issue Type: New Feature
>          Components: contrib - DataImportHandler
>            Reporter: Mikhail Khludnev
>            Priority: Minor
>              Labels: DIH, dataimportHandler, dih
>         Attachments: SOLR-4799.patch, SOLR-4799.patch, SOLR-4799.patch, SOLR-4799.patch,
SOLR-4799.patch
>
>
> DIH is mostly considered as a playground tool, and real usages end up with SolrJ. I want
to contribute few improvements target DIH performance.
> This one provides performant approach for joining SQL Entities with miserable memory
at contrast to http://wiki.apache.org/solr/DataImportHandler#CachedSqlEntityProcessor  
> The idea is:
> * parent table is explicitly ordered by it’s PK in SQL
> * children table is explicitly ordered by parent_id FK in SQL
> * children entity processor joins ordered resultsets by ‘zipper’ algorithm.
> Do you think it’s worth to contribute it into DIH?
> cc: [~goksron] [~jdyer]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message