manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Job Multiple Outputs
Date Tue, 10 Sep 2019 16:09:05 GMT
The output connection contract is that a request to index is made to the
connector, and the connector returns when it is done.
When there are multiple output connections, these are each handed a copy of
the document, one after the other, and told to index it.  This is all done
by one worker thread.  Multiple worker threads are not used for multiple
outputs of the same document.

The framework is smart enough to not hand a document to a connector if it
hasn't changed (according to how the connector computes the
connector-specific output version string).

Karl


On Tue, Sep 10, 2019 at 11:00 AM Julien Massiera <
julien.massiera@francelabs.com> wrote:

> Hi,
>
> I would like to have an explanation about the behavior of a job when
> several outputs are configured. My main question is : for each output,
> how is the docs ingestion managed ? More precisely, are the ingest
> processes synchronized or not ? (in other words, is the ingestion of the
> next document waiting for the current ingestion to be completed for both
> outputs ?). But also, if one output is configured to send a commit at
> the end of the job, is this commit pending until the last ingestion has
> occured in the other output ?
>
> Thanks for your help,
> Julien
>

Mime
View raw message