As requested: (TL;DR) feeding a family with two children is technically feasible on the income in your offer letter; EPFL's PhD/Postdoc union (ACIDE) can help answer a lot of your more detailed questions.
As a foreign employee, unlike Swiss citizens, your income will be taxed at source, meaning that your take home pay will be somewhere around 41K CHF/year (I don't have the precise number available, so you will need to do some research to find the exact tax rate for your income level). Now, with two kids, you will probably receive somewhere between 5000 - 10000 in "allocation familiales" (again, I don't have the precise number available, if you contact the professor who gave you the offer, his secretary can probably find out from HR for you). You can I think count on around at least 46K take home pay.
Now, for expenses:
A two bedroom apartment in the Lausanne area runs (depends on location and size) anywhere between 1400 -- 3000 per month. In Switzerland (for complicated reasons) the landlords will be likely quite picky. With 50K/annum income you can likely only get accepted at an apartment that cost right on the low end (1.4K - 1.5K). Luckily, unlike regular employees, as a PhD student you can qualify for housing designated to be rented out at "student rates" (my understanding is that the landlord gets some tax benefits from that). Unfortunately for you, most of those are studio or one bedroom apartments. You will have to depend a bit on luck here.
If your wife doesn't work, you don't have to worry about day care for the two kids.
Another large expense which you may not be aware of is health insurance. Recently the EPFL ACIDE (union of "intermediate" employees; meaning mostly PhD and postdocs) has managed to secure preferred rates for health insurance. Without it you can expect to pay around 14K per year for the mandatory health insurance for your whole family. But under the preferred rate you can get away with around 7K or less (the health insurance premium depends on your home address; if you live in a wealthier neighborhood you are expected to pay more) for two adults and two kids.
With your family size you will probably pay around 1500 for food and utilities (TV/internet/water/local garbage tax etc) per month; again, if you shop at bargain stores like Aldi instead of the local Swiss stores like Migros or Coop, you can probably spend as little as half that.
So overall, I think that with your family size you should be able to just get by, and possibly save a little money, if you really watch your expenditures (don't spend money on unnecessary things, shop for deals in groceries etc.) It may be tight, but doable.
The hard thing will be finding housing. The university proper cannot help that much. Your future professor's secretary, however, should have access to the internal posting as well as the ACIDE postings. That can be a good place to look. (Actually, you should look around on the ACIDE website, since there are a lot of useful information there. And if you e-mail them, they may be able to find out the current numbers for you to replace the ones I quote here from memory.)
The university does not help out with employment for the spouse (not even for permanent employees in general when the spouse is not in academia). I am not sure if it would be a good idea for your spouse to work: daycare is really, really expensive. I don't know your situation, but if you have two toddlers it can be easily in the range of 5000 CHF/month, which would be likely more than a starting income for whatever job your spouse can find. Now, there are socially subsidized daycare in the greater Lausanne area (I think at least one or two of those are on campus; if I remember correctly the first two on this list have available socially subsidized daycare positions). For the subsidized daycare options the charge is progressively proportional to your income. As a PhD student in Lausanne with 51K salary, you will pay very little for childcare (as little as 1500 - 2000 for full time care for two kids)... provided you get a spot. The waiting list can be as long as 18 months (though since you are "low income" you will get prioritized somewhat; but unless your spouse also has a job, there will be people ahead of you).
TA'ing will not get you extra income, as far as I know. PhD students' employment contracts usually have teaching assistantships written in as part of the time commitment, unless your professor has funds to fully pay you 100% for research. (If you are not sure, you can e-mail the professor to ask what your expected split between teaching/research is.) If your professor is paying you full time for research, he or she will not be that happy with you spending time on other employment.
If you do take the position: the public transportation system in Switzerland is very reliable and relatively affordable (compared to everything else there). The upside is that you don't have to live in the city even if you don't have a car, so if you see a reasonably priced apartment in a village somewhere, don't be discouraged by the commute. And as a PhD student at EPFL, you will be officially an employee of the Swiss Federation, so you can get discounted rates for train tickets and such (ask the secretary about the demi-tarif).
Lastly, if you don't intend to stay in Switzerland: one thing people often overlook is that by law you are required to contribute the the deuxieme pillar (part of the Swiss retirement fund system). The basic social security, like taxes, is lost to you. But the second pillar (as a PhD student over 3 years you will contribute probably just under 10K CHF) can be withdrawn if you leave Switzerland permanently to a non-EU country, or transferred to an equivalent retirement savings account if you go to a EU country. So even though money may be on the tight side while you are a student, you will come out with a little bit of savings, as opposed to absolutely zero.