spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bin Wang <wbi...@gmail.com>
Subject Checkpoint directory structure
Date Wed, 23 Sep 2015 10:58:34 GMT
I find the checkpoint directory structure is like this:

-rw-r--r--   1 root root     134820 2015-09-23 16:55
/user/root/checkpoint/checkpoint-1442998500000
-rw-r--r--   1 root root     134768 2015-09-23 17:00
/user/root/checkpoint/checkpoint-1442998800000
-rw-r--r--   1 root root     134895 2015-09-23 17:05
/user/root/checkpoint/checkpoint-1442999100000
-rw-r--r--   1 root root     134899 2015-09-23 17:10
/user/root/checkpoint/checkpoint-1442999400000
-rw-r--r--   1 root root     134913 2015-09-23 17:15
/user/root/checkpoint/checkpoint-1442999700000
-rw-r--r--   1 root root     134928 2015-09-23 17:20
/user/root/checkpoint/checkpoint-1443000000000
-rw-r--r--   1 root root     134987 2015-09-23 17:25
/user/root/checkpoint/checkpoint-1443000300000
-rw-r--r--   1 root root     134944 2015-09-23 17:30
/user/root/checkpoint/checkpoint-1443000600000
-rw-r--r--   1 root root     134956 2015-09-23 17:35
/user/root/checkpoint/checkpoint-1443000900000
-rw-r--r--   1 root root     135244 2015-09-23 17:40
/user/root/checkpoint/checkpoint-1443001200000
drwxr-xr-x   - root root          0 2015-09-23 18:48
/user/root/checkpoint/d3714249-e03a-45c7-a0d5-1dc870b7d9f2
drwxr-xr-x   - root root          0 2015-09-23 17:44
/user/root/checkpoint/receivedBlockMetadata


I restart spark and it reads from
/user/root/checkpoint/d3714249-e03a-45c7-a0d5-1dc870b7d9f2. But it seems
that the data in it lost some rdds so it is not able to recovery. While I
find other directories in checkpoint/, like
 /user/root/checkpoint/checkpoint-1443001200000.  What does it used for?
Can I recovery my data from that?

Mime
View raw message