say there's some logs:

s3://log-collections/sys1/20141212/nginx.gz
s3://log-collections/sys1/20141213/nginx-part-1.gz
s3://log-collections/sys1/20141213/nginx-part-2.gz

I have a function that parse the logs for later analysis.
I want to parse all the files. So I do this:

logs = sc.textFile('s3://log-collections/sys1/')
logs.map(parse).saveAsTextFile('s3://parsed-logs/')

BUT, this will destroy the date separate naming shema.resulting:

s3://parsed-logs/part-0000
s3://parsed-logs/part-0001
...

And the worse part is that when I got a new day logs. 
It seems rdd.saveAsTextFile couldn't just append the new day's log.

So I create a RDD for every single file.and parse it, save to the name I want.like this:

one = sc.textFile("s3://log-collections/sys1/20141213/nginx-part-1.gz")
one.map(parse).saveAsTextFile("s3://parsed-logs/20141213/01/")

which resulting:
s3://parsed-logs/20141212/01/part-0000
s3://parsed-logs/20141213/01/part-0000
s3://parsed-logs/20141213/01/part-0001
s3://parsed-logs/20141213/02/part-0000
s3://parsed-logs/20141213/02/part-0001
s3://parsed-logs/20141213/02/part-0002

And when a new day's log comes. I just process that day's logs and put to the proper directory(or "key")

THE PROBLEM is this way I have to create a seperated RDD for every single file. 
which couldn't take advantage of Spark's functionality of automatic parallel processing.[I'm trying to submit multi applications for each batch of files.]

Or maybe I'd better use hadoop streaming for this ?
Any suggestions?