Purchased the Humble Indie Bundle V. Downloaded Bastion (not through Steam) and on startup, an error dialog popped up.

Turns out the .NET Framework was somehow disabled.

Windows 7:
Control Panel, Programs and Features, Turn Windows features on or off
Microsoft .NET Framework 3.5.1 - Make sure this is checked.