What do you think about having one "free pass" in marriage?

My husband and I have been married for 6 years, together for 9 years. We have two kids - one three-year-old and a one-year-old. Ever since I got pregnant with the second, our sex life has ground to a halt. He gained some sympathy pregnancy weight that made his already big frame - very big. He's a lot bigger than when we first got married and, to be honest, I don't find it attractive. I can't even muster enough energy or interest to even pretend. However, part of me feels desperate to have something of me when I was younger - to just have one free pass to have an unadulterated passionate fling. The fantasies are endless. I told him his excess weight was a problem for me to get "in the mood." He's trying to lose weight but he's been trying for longer than I've known him - it's just the past year the weight has gotten out of control. I even asked for a free pass (which he said no). I don't want to hurt him, as I love him, despite how this might sound. I'm just a sexual woman - I enjoy it - and want to have those feelings again with someone other than amusing myself. In the end I'll probably not do anything to jeopardize the marriage but unless he does something drastic about the weight, I've zero interest in having sex with him. Has anyone been in this position and what did you do?