pig-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nishanth S <nishanth.2...@gmail.com>
Subject Delete hdfs directory afterpig execution
Date Wed, 30 Jan 2019 23:47:38 GMT
Hey folks,
I have a  pig script as below. I want to delete a directory after  data is
moved   from location x to y as you see below. I then want to delete the
directory .However the fs command  throws an error .Is this the correct
way  to delete the directory .Also is  there a guarantee that the fs
command will be executed in order ?

SET pig.maxCombinedSplitSize 268435456;
A = load '$SOURCE' using org.apache.pig.piggybank.storage.avro.AvroStorage
();
SET mapred.output.compress true
SET mapred.output.compression.codec
org.apache.hadoop.io.compress.SnappyCodec
SET avro.output.codec snappy
store A into '$TARGET' using
org.apache.pig.piggybank.storage.avro.AvroStorage ();
fs -rm -r '$SOURCE'

Thanks,
Nishanth

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message