This was originally committed as f8452ddfcc33 and reverted in 7cb1aa9d9368.
The issue was that shell builtins were being escaped too, and apparently
Bash won't execute a builtin when it is quoted e.g. '!'. Instead, it
thinks it's a command and it can't find it.
Re-committing the change with that issue fixed.