Compare where you want to life or travel. If you want to be in a bigger city like Berlin than it is not important to speak German because almost everyone is kind of international there and in almost every community you find people from overseas or people who speaks English. But if you want to travel through the country and visit the smaller villages and stuff like that you definitely need to have some kind of basic Germany skills.
 
P.S.
What I also hear from many friends who speaks only English and start to learn German, when you ask something in German (of course you have an accent) you mostly getting an English answer back. So don’t be upset with that. ;)