kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Young, Ben" <Ben.Yo...@fisglobal.com.INVALID>
Subject Copy topic to offline file
Date Thu, 07 Jun 2018 17:47:52 GMT
Hi

We have a requirement that we need to be able to copy a portion of a Kafka topic to a file
(perhaps between a range of timestamps) which can then be loaded into a Kafka instance on
a different machine. This machine may have a different number of partitions etc for the topic.
We can't connect them via connect or anything, this is more for offline analysis of data that
produce issues in our software, or for providing data for regression tests.

I'm happy to write a tool to do this (I think the OffsetsForTimestamp API does what I need),
but does anyone know of any tools out there that do something like this? I've had an initial
google, but I can't find anything

Thanks,
Ben

Ben Young
Principal Software Engineer - Risk
Risk & Compliance
T:  +44 (0) 1865 578140
E: ben.young@fisglobal.com
FIS | Empowering the Financial World
FIS Systems Limited * Registered in England and Wales No. 1937159 * Registered Office: Level
39, 25 Canada Square, London E14 5LQ

The information contained in this message is proprietary and/or confidential. If you are not
the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose,
distribute or use the message in any manner; and (iii) notify the sender immediately. In addition,
please be aware that any message addressed to our domain is subject to archiving and review
by persons other than the intended recipient. Thank you.

Mime
View raw message