My questions is regarding forms and if they are different because he is a refugee?
There may be some differences because of your husband's refugee status, but I expect they will be minor. You can start at http://www.austria.org/residence-permit. The category that applies to you is II) Niederlassungsbewilligung (for foreigners who intend to live in Austria for a longer period of time = Permanent Residence Permit) (in particular, "family members of aliens living in Austria, family re-unification").
Does this mean I give up my US citizenship?
Will I be able to travel from Austria to America without a problem, in emergencies or holidays?
Yes. A US citizen cannot be denied entry into the US. As long as you hold an Austrian residence permit, you can leave and re-enter Austria at will.
We are not married yet, can I go to austria, marry him and then do the paperwork?
I'm not sure about this; it will depend on Austria's laws concerning marriage. Most jurisdictions will allow a couple to marry if at least one of them resides in the jurisdiction. Once you are married, however, it may be necessary to return to the US to apply for the permit, but it may also be possible to apply for the permit as a fiancée before you travel to Austria. It is not entirely clear from the website.
Where do I begin and what forms do I need to fill out?
Is there a place I can go to get one on one help with this matter?
You can try asking at the embassy or consulate. Your fiancé can try asking for help at the migration agency that placed him in Austria or any other Austrian organization that offers support to refugees. If you can afford it, you can hire an Austrian immigration lawyer.
Lastly, do I need to speak German?
Probably. The austria.org page linked above has a section on the language requirement. Certain categories of immigrants are exempt from the language requirement, but from what you've said in your question it seems that you will not be.