I'm self employed American, currently living in the US, and have the possibility of becoming an Austrian citizen. Once I do this can I move to and live permanently in the UK (and continue to be self employed there)? What are the delays involved or residence requirement involved? Can I for example, move to the UK and live in the UK indefinitely immediately after acquiring Austrian citizenship?
Yes you can. Having an EU passport, you're allowed to freely move, stay and live permanently in any other EU country (there were some exceptions with recent countries joined in EU but all these have been lifted now).
But it's a good idea to check Austrian naturalization laws and ensure that they don't have any T&Cs for their citizenship. For example, in Ireland if you get Irish citizenship and you're not intending to stay permanently in Ireland, they have the right to revoke your citizenship (http://www.citizensinformation.ie/en/moving_country/irish_citizenship/becoming_an_irish_citizen_through_naturalisation.html) unless you go through certain "hoops"
You can continue to reside in the UK as long as you fulfill the conditions (I don't know exactly what they are for self-employed people, I suppose some genuine activity is required, merely registering yourself as self-employed might not be enough) or qualify in some other way (e.g. take up employment). After 5 years under this regime you would become a permanent resident and, under current EU rules, you could then stay in the UK indefinitely even if you give up your activity.
As far as I know, the UK does not require any registration for EEA citizens. It is therefore not necessary to provide any proof of your status (although the UK could require it under current EU rules and there is some talk of introducing something like that).