Hi, On Fri, Dec 6, 2013 at 9:38 AM, Tom Brown wrote: > Do you mean you want to start from the most recent data and go backwards to > the oldest data, or that you want to start with old data and consume > forwards? > Forwards is the "normal way". I'm looking for the "abnormal way", of course ;) i.e. backwards. If the following are the messages that came in, oldest to newest: M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 Then I'd love to be able to consume from the end, say in batches of 3, like this: get last 3: M10 M11 M12 get last 3: M7 M8 M9 get last 3: M4 M5 M6 get last 3: M1 M2 M3 Of course, if messages keep coming in, then the new ones that arrive would get picked up first and, eventually, assuming Consumer can consume faster than messages are produced, all messages will get consumed. But the important/key part is that any new ones that arrive will get picked up first. If the former, it would be difficult or impossible in 0.7.x, but I think > doable in 0.8.x. (They added some sort of message index). If the latter, > that is easily accomplished in both versions. > I'd love to know if that's really so and how to do it! We are looking to move to Kafka 0.8 in January and to add performance monitoring for Kafka 0.8 to SPM (see http://blog.sematext.com/2013/10/16/announcement-spm-performance-monitoring-for-kafka/) Thanks, Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ > On Friday, December 6, 2013, Otis Gospodnetic wrote: > > > Hi, > > > > Does Kafka offer a way to consume messages in batches, but "from the > end"? > > > > This would be valuable to have in all systems where the most recent data > is > > a lot more important than older data, such as performance metrics, and > > maybe even logs....maybe also trading/financial data, and such. > > > > Any chance of this sort of functionality ever making it into Kafka, or is > > this simply not implementable due to some underlying assumptions, or data > > structures, or ... ? > > > > Thanks, > > Otis > > -- > > Performance Monitoring * Log Analytics * Search Analytics > > Solr & Elasticsearch Support * http://sematext.com/ > > >