You have a right to live in France under EU rules if you qualify in one of four ways: working, having sufficient resources, studying or being the spouse of another EU citizen who qualifies. The requirements are detailed on the page on the “carte de séjour UE” on service-public.fr (in French). In fact, applying for this “permit” is not mandatory but you need to fulfill the very same requirements to have a right to reside in France, even if you forgo applying for an actual physical carte.
In your case, as a retiree, this means that you need to have “sufficient resources”, defined as above the threshold to qualify for welfare benefits. In France, for someone above the age of 65, you cannot in any case be required to have more than €9600 per year for one person (or €14904 for a couple) but it might even be possible to qualify with a lower income in some cases. These rules are based entirely on citizenship, having been a resident in the UK before moving to France is not a requirement.
If you do not qualify, you could probably move to France as a British citizen anyway because nobody bothers with tracking all EU citizens present in the country and you are not at risk of a fine or anything like that. But under EU rules, the French authorities could in principle ask you to leave at any time and you do not have any unconditional right to reside in the country. Most importantly for you, you would not be able to sponsor your wife (more on that below).
Regarding healthcare, as a retiree from elsewhere in the EU, you would not automatically be entitled to enrollment in the government's “universal coverage” scheme and having health insurance is actually one of the requirements to qualify for residence as “inactive person”.
But if you would qualify for statutory healthcare coverage in the UK, you should be able to transfer those rights to France. I don't know the UK system well enough to know if you would qualify and how to go about it but since you get a UK government pension, I would guess that it should be possible. In practice, what you need is called the “S1 document”. I found some info about that on nhs.uk and a forum for British expats in France.
Failing that, you would probably be required to get private insurance coverage. Since healthcare costs are lower in France, it should be cheaper than what you could get in the US but you still need to pay for it.
Your wife is in a very different situation. As a non-EU citizen, she needs a visa or residence permit to stay longer than three months and she does not have a right to reside in France on her own. So while holding a carte de séjour was optional for you, it is mandatory for her. Here as well, citizenship, not residence, defines her position. Having been a resident in the UK before moving to France would not generally make things much easier.
But if you, as a British citizen, reside in France then she can get a residence permit “membre de la famille d'un citoyen de l'Union” relatively easily (that's more-or-less the French equivalent to the “EEA family permit” mentioned by Gayot Fow). She will have to prove her relationship with you (e.g. through a marriage certificate), your right to stay in France (e.g. prove that you fulfill the financial means requirement discussed above) and pay a small fee.
I am not entirely sure of her health insurance situation. If you are covered through one of the statutory health insurance schemes in France, she would also automatically be covered at no extra cost. But I don't know if this also applies if you qualify through the S1 document (I think it should). Private insurers have their own rules and will almost certainly require extra payments to cover her.