Well that was a struggle! I eventually found:
“Without prejudice to state provisions, sufficient living space is usually deemed to be available if 12 sqm of living space are available to each family member aged above six and 10 sqm of living space for each family member aged below six. Adequate availability of the facilities (kitchen, bathrooms, lavatories) must be ensured. A shortfall of about ten per cent is acceptable” (2.4.2. General Administrative Regulation to the Residence Act).
That is a quote from this PDF on the BAMF (Bundesamt für Migration und Flüchtlinge) website.
There will be a total of five of you, so with a 100 sqm house you will have 20 sqm each. The only possible catch is what "100 sqm" means. It is quite common in Germany for not all space within an apartment to count - toilets/bathrooms may not count at all, kitchens/balconies count half, and living rooms/bedrooms are fully counted. However it is unlikely that will bring you under the 12 sqm figure; also, if you got the 100sqm from the property details (rather than your husband getting out a tape measure), they will already have done that pro-rating.