It's a tricky one, the law is not that specific. A full answer, if it is possible, would require a good knowledge of case law that I do not possess. What is clear is that you must continue to “fulfill the conditions for issueing for the residence permit”. In most cases (work, studies), it's difficult to do that while being abroad making the question moot, which might be why it's not addressed directly.
In your case, the key requirement is actually sharing your wife's life (it's called “communauté de vie”). For example, if you decide to separate, she cannot sponsor you. But the exact meaning of that concept is set by the courts and they have lot of discretion. I came across one court decision where someone failed to appeal a residence permit refusal because they chose to live in another part of France than their spouse for work. The decision however did not mention anything about weekends. If you are able to show that your main residence is still Paris and you go there as often as you can, you might still fulfill the “communauté de vie” requirement.
