I've gone through that in the other way, while leaving Germany. Under new regulations in EU, you are obliged to have health insurance in any EU country, which means you are not allowed to unregister until you prove you've got another health insurance.
On the other way, you are not allowed to voluntarily register when you have health insurance in other country (in my case, it meant, I was stuck with expensive German health insurance until I find job in Poland, which I solved by registering a company and getting cheap insurance from it).
It means, if you're from EU, and you don't work or run a company in Germany, you're insured with your old insurance. If you get a job, things got more tricky. In Poland you would be automatically insured by your employer, but German law gives you the option: either you'll be insured by your employer, or you get insured private, in which case you should ask, if it need to be German insurance.
But you must take into account, health insurance in other EU countries is meant to work in emergency cases, not on regular basis. Each case can be checked, if it was necessary for you to use health service abroad, or you could as well go back to your country. You can find it difficult to get sick leave. You won't have German electronic insurance card, which is your password to quick access to your doctor.