You may want to look at http://ec.europa.eu/dgs/home-affairs/what-we-do/policies/legal-migration/long-term-residents/index_en.htm as well; this is the page describing long term residency for the general public rather than for a law audience.
The German permanent resident is entitled to a residence permit in another EU country, but does not immediately become a permanent resident in that country. Instead, the permanent resident retains a right to return to Germany. After living in the second country for five years, the resident can apply for permanent residence in the second country. When that status is granted, the German permanent residence ceases.
If the person remains outside the state of permanent residence for six years, the right of permanent residence is lost, even if it has not been acquired elsewhere.
This is controlled by directive 2003/109/EC, especially chapter III. Chapter III also provides for certain conditions that can be imposed for the issuance of a residence permit.
A few EU countries have opted out of this directive, so this answer does not apply to those countries. As noted in paragraphs 25 and 26 of the preamble, the countries opting out of the directive are the United Kingdom, Ireland, and Denmark.