You will most likely be entitled to a long-term visa, but that does not necessarily give you the right to work in France. It does however relieve you of the need to worry about leaving the country periodically.
(If you are not able to secure a long-term visa, then your presence in France will be limited to 90 days in any 180-day period, so yes, you'd have to leave after 90 days and remain outside the Schengen area for 90 days before you can reenter. This has nothing to do with whether you are working or not; it has only to do with your presence in the Schengen area.)
Immigration law has emphatically not kept pace with remote work. Even if your visa comes with a right to work, I think there's actually no legal way for you to do what you're proposing unless your company has a legal presence in France and can hire you as a French employee.
For example, French labor law gives employees much greater protection than US labor law. It is probably illegal for your company to employ you in France without giving you these protections. Additionally, the law probably calls for your employer to pay French payroll taxes, which your employer is unlikely to be set up for.
The easiest way to deal with these considerations is probably for you to set up a French company to work for your employer as a contractor.
As for tax implications, your absence from the US might qualify you for the foreign earned income exclusion. You would probably be liable to pay French income tax even if you are getting paid in the US. Obviously, if you are set up as an employee or a contractor getting paid in France, you will certainly be liable for French income tax.