DataFrames are a narrower, more specific type of abstraction, for tabular data. Where your data is tabular, it makes more sense to use, especially because this knowledge means a lot more can be optimized under the hood for you, whereas the framework can do nothing with an RDD of arbitrary objects. DataFrames are not somehow a "better RDD".
Datasets are more like the new RDDs, supporting more general objects and programmatic access. Still a different thing for a different purpose from DataFrames. But has an API more similar to DataFrames and some of the same types of benefits for simple types via Encoders.