kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Blender Bl <bl3nd...@aol.com>
Subject Persist Queue On HDFS
Date Tue, 14 Jan 2014 21:16:25 GMT

Hi,


My team trying to implement lambda architecture.
We need to stream all our new data though Kafka to storm, and HDFS.


As i see it were are two options:

Using Camus - not very efficent
Streaming via Storm - not very efficent

Is it possible to persist the queue's files over the HDFS (with short-circuit setting switched
on) instead of the local filesystem?


It suppose to have similar performance to the local file system, doesn't it?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message