hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "lynn (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HAWQ-1552) hawq does not support hdfs storage policy?
Date Fri, 17 Nov 2017 09:35:00 GMT
lynn created HAWQ-1552:

             Summary: hawq does not support hdfs storage policy?
                 Key: HAWQ-1552
                 URL: https://issues.apache.org/jira/browse/HAWQ-1552
             Project: Apache HAWQ
          Issue Type: Bug
          Components: libhdfs
            Reporter: lynn
            Assignee: Radar Lei

1.The '/ssd" path on HDFS set storage policy ALL_SSD.
hdfs storagepolicies -setStoragePolicy -path /ssd -policy ALL_SSD

2. check
[hdfs@master1 ~]$ hdfs storagepolicies -getStoragePolicy -path /ssd
The storage policy of /ssd:
BlockStoragePolicy{ALL_SSD:12, storageTypes=[SSD], creationFallbacks=[DISK], replicationFallbacks=[DISK]}

3. put file to hdfs
hdfs dfs -put dd.txt /ssd/fs_ssd

4.check block location
[hdfs@master1 ~]$ hdfs fsck /ssd/fs_ssd/dd.txt -blocks -locations -files
decommissioned replica(s) and 0 decommissioning replica(s).
0. BP-845848702- len=7 repl=3 [DatanodeInfoWithStorage[,DS-1510d4e4-cfdb-4184-8f47-7417b91f4f5c,{color:red}SSD{color}],

5.hawq create filespace fs_ssd and tablespace ts_ssd, fs_ssd point to /ssd/fs_ssd path 

6.psql create table 
create table p(i int)with(appendonly=true,orientation=parquet,compresstype=snappy)  tablespace

7. psql insert data
insert into p values(1);

8. query the file on hdfs
select c.relname, d.dat2tablespace tablespace_id, d.oid database_id, c.relfilenode table_id
  from pg_database d, pg_class c, pg_namespace n
 where c.relnamespace = n.oid
   and d.datname = current_database()
   and c.relname = 'p';

relname | tablespace_id | database_id | table_id 
 p       |       1021474 |     1021475 |  1037187

9. check the file of table "p" locations
[hdfs@master1 ~]$ hdfs fsck /ssd/fs_ssd/1021474/1021475/1037187/1 -blocks -locations -files
Connecting to namenode via http://master1.bigdata:50070/fsck?ugi=hdfs&blocks=1&locations=1&files=1&path=%2Fssd%2Ffs_ssd%2F1021474%2F1021475%2F1037187%2F1
FSCK started by hdfs (auth:SIMPLE) from / for path /ssd/fs_ssd/1021474/1021475/1037187/1
at Fri Nov 17 17:26:17 CST 2017
/ssd/fs_ssd/1021474/1021475/1037187/1 188 bytes, 1 block(s):  OK
0. BP-845848702- len=188 repl=3 [DatanodeInfoWithStorage[,DS-4be28698-6ebd-4ae0-a515-f3fb5e1293ab,{color:red}DISK{color}],

The ALL_SSD storage policy doesn't work!!!

This message was sent by Atlassian JIRA

View raw message