storm-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From da...@apache.org
Subject [08/12] storm git commit: updated README.md
Date Fri, 14 Aug 2015 18:01:24 GMT
updated README.md


Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/4d8c51b4
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/4d8c51b4
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/4d8c51b4

Branch: refs/heads/master
Commit: 4d8c51b4503110f03a27b401f096ff2477bbb7cf
Parents: cdabbb3
Author: Arun Mahadevan <aiyer@hortonworks.com>
Authored: Mon Aug 10 14:01:24 2015 +0530
Committer: Arun Mahadevan <aiyer@hortonworks.com>
Committed: Mon Aug 10 14:01:24 2015 +0530

----------------------------------------------------------------------
 external/storm-hdfs/README.md | 3 +++
 1 file changed, 3 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/storm/blob/4d8c51b4/external/storm-hdfs/README.md
----------------------------------------------------------------------
diff --git a/external/storm-hdfs/README.md b/external/storm-hdfs/README.md
index 98fcc55..e1f2b28 100644
--- a/external/storm-hdfs/README.md
+++ b/external/storm-hdfs/README.md
@@ -323,6 +323,9 @@ duplicates from the current data file by copying the data up to the last
transac
 operation involves a lot of data copy, ensure that the data files are rotated at reasonable
sizes with `FileSizeRotationPolicy` 
 and at reasonable intervals with `TimedRotationPolicy` so that the recovery can complete
within topology.message.timeout.secs.
 
+Also note with `TimedRotationPolicy` the files are never rotated in the middle of a batch
even if the timer ticks, 
+but only when a batch completes so that complete batches can be efficiently recovered in
case of failures.
+
 ##Working with Secure HDFS
 If your topology is going to interact with secure HDFS, your bolts/states needs to be authenticated
by NameNode. We 
 currently have 2 options to support this:


Mime
View raw message