kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shiti Saxena <ssaxena....@gmail.com>
Subject Re: Programmable API for Kafka Connect ?
Date Thu, 07 Jan 2016 08:07:15 GMT
Hi Alex,

After going through the documentation, I realized that what I was actually
referring to is the "Embedded API" discussed in KIP-26. Is KAFKA-2378
<https://issues.apache.org/jira/browse/KAFKA-2378> the related issue? If
so, has there been any progress on this or can I work on that?

Thanks,
Shiti

On Wed, Jan 6, 2016 at 11:21 PM, Alex Loddengaard <alex@confluent.io> wrote:

> Hi Shiti,
>
> In the context of your question, there are three relevant components to
> Connect: connectors, tasks, and workers.
>
> Connectors do a number of things but at a high level they specify the tasks
> (Java classes) to perform the work, how many to start, and some
> coordination. The tasks do the actual work of reading/writing data to/from
> Kafka (using the Connect API, not producer/consumer API). And the workers
> are the daemons that run on one or many nodes that run the connectors and
> tasks (in distributed mode).
>
> The only way to start worker processes is through the command line --
> bin/connect-distributed.sh.
>
> The only way to add/remove/modify connectors in distributed mode is through
> the REST API. I don't know of any API clients, unfortunately, but maybe one
> exists that I don't know of? The REST API is started along with the worker
> process, as part of bin/connect-distributed.sh. Meaning, you don't have to
> start a separate REST process.
>
> Let me know if you have any follow-up questions, Shiti.
>
> Alex
>
> On Tue, Jan 5, 2016 at 9:12 PM, Shiti Saxena <ssaxena.ece@gmail.com>
> wrote:
>
> > Hi,
> >
> > Does Kafka Connect have an API which can be used by applications to start
> > Kafka Connect, add/remove Connectors?
> >
> > I also do not want to use the REST API and do not want to start the REST
> > server.
> >
> > Thanks,
> > Shiti
> >
>
>
>
> --
> *Alex Loddengaard | **Solutions Architect | Confluent*
> *Download Apache Kafka and Confluent Platform: www.confluent.io/download
> <http://www.confluent.io/download>*
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message