In general, residency depends on a particular purpose. There is no absolute sense in which someone is or isn't a resident of a particular place. As you note in the question, taxation is one such purpose; other purposes include immigration, driver licensing, and voting.
Is a resident for tax purposes automatically a resident for immigration purposes?
No. For example, consider someone who maintains a home in Canada and spends about half of each year there. The rest of the time is spent traveling in the US, as a B-1 business visitor, to several different cities, and the time there is enough that the person passes the substantial presence test. That person is resident of the US under US tax law, but probably for no other purpose. (In fact, such a person is likely to be a tax resident of both Canada and the US under each country's laws.)
If somebody has obtained a temporary or permanent residence permit of a country, but does not "live" there, is he a resident of that country?
For immigration purposes, possibly, unless the person's absence causes the immigration status to lapse. But being a resident in that country under that country's immigration law does not mean that other countries would consider that person a resident of the country, and it doesn't stop anyone from considering the person to be resident in two places at once. For purposes other than immigration, having a residence permit probably doesn't have much effect (a counterexample to this is that US permanent residents are treated as US tax residents no matter where they physically reside).
If someone is living in a country illegally, would that be considered his country of residence?
For some purposes, probably. For example, the US substantial presence test applies to people who are in the US unlawfully, so such a person becomes a tax resident of the US at a certain point. For other purposes, probably not. For example, Schengen visa applications are supposed to be submitted to the consulate responsible for a person's place of residence, but the person must be legally resident there, so an Ethiopian living illegally in Ecuador would have to go to Ethiopia to apply for a Schengen visa.
Edit: for greater clarity, this is for immigration purposes. Some questions that are asked:
- provide details for the countries that you have lived in for last 10 years
Note the use of the phrase "lived in." It could be argued that a person with a permanent resident permit for one country who spends 90% of the time at a home in another country is at least in some sense a resident of the first country, but it's not reasonable to say that that person "lives in" the first country.
Whether you "lived in" a given place would normally be determined by your physical presence, or, if you are itinerant, by your home base (if you have one). Since the question is "provide details for...," your response can include some explanation if that is necessary. For example, if you were traveling around North America as a business consultant, but without a home base in any country, you could say as much in your response.
- police record certificates for countries that you have lived in for longer than six months
This would also depend on your physical presence. Assuming you have no police record, if you're in doubt about whether to include a country as one you have "lived in" then you should include it. Omitting it could lead to a suspicion that you are trying to be deceptive, while including it would have no negative effect. If you do have a police record in the questionable place then you should ask a lawyer before submitting the application without the certificate from that place.