I've asked some colleagues who have had experience with re-entering Canada without the piece of paper and the answer is no, it's not mandatory. At the border you will (at worst) be sent to secondary screening where the immigration officer will look up your status. And occasionally they will simply let you in under the ETA without verifying the work status.
Update: Tried it myself this week - worked without any issues at the land border. The immigration officer checked the computer to see if my work permit is still valid and then let us through. Didn't even have to go to secondary screening.
Update #2: Over the past 10 months I've personally witnessed at least five colleagues who have forgotten their work permit, both driving and taking the bus - no one ever had any issues whatsoever. Not sure why CBSA can't just drop the requirement altogether.