Following a Unity Rigidbody2D with a camera works perfectly if done right. If not done right, it's a frustrating mess.
Unfortunately, knowing what constitutes "done right" is also a frustrating mess, and interpolation issues in the 4.5.x and 4.6.1 releases made it difficult to identify the correct technique.
Here's what you need to know:
Use Update or LateUpdate, not FixedUpdate
A lot of people suggest using FixedUpdate to maintain your camera position, and it's true that only updating your camera on physics updates results in smooth following. But GameObjects not controlled by RigidBody2D components will then be seen to stutter. The only solution is to use Update, and make Unity properly update the target's position using interpolation or extrapolation.
Use Interpolation or Extrapolation to smooth the target's position between physics frames
This produces smooth inter-FixedUpdate positions, with caveats. Prefer interpolation, because its degradation in case of extreme velocity changes will be lag, not overshoot.
Set interpolation at the correct time
As of Unity 4.6.1, you need to set interpolation on a Rigidbody2D before Update runs on it. If you do it during or after the first Update(), the property changes without having any effect on the Rigidbody2D. This is probably a bug.