Hadoop 0.20.1 is finally here. Get it while it's hot! If you want, you can read the full release notes. This is the release to use if you are setting up a new cluster. It's also worth upgrading older pre 0.20.x clusters to this release.
Hadoop 0.20.x is very different from previous releases. The configuration and APIs have been overhauled. As previously mentioned, there is the new TFile storage format.
Look for an imminent release of PIG 0.4 release and Cloudera distribution CDH2 0.20.x with Hive and PIG support.