Okay, I have a couple suggestions on things for you to try, as well as a potential compromise. Of course, you are the expert on your little one, so if one of the suggestions doesn’t make sense to you, ignore it 😉
I do know moms who have had success by simply moving the baby out of their room. Right now, you are there to hear every sound he makes, and he hears every sound/stirring that you (and your husband) make. I resisted this change for awhile, but it did end up improving things for you a bit. I totally get why you don’t want to make this change, but you may be surprised.
Have dad handle all night wakings. You are your guy’s favorite (sorry Daddy). We also found that it was a clear signal to our little man that he wasn’t going to get fed. If I was going to feed him, I’d go in after a short “fuss” period (3-4 minutes, just in case he’d put himself back to sleep). If I wasn’t, daddy would go in, give snuggles, say “no milk”, and put him back down. You could have DH get him all the way back to sleep, or let him do some fussing, but we always figured this was a clearer sign that mommy wasn’t coming (rather than mommy can’t hear me…I will yell louder).
Try bottles only at night. This “less desired” feeding method will mean your little man will drink if he’s hungry, but not if he isn’t. This for us went along with “dad does night wakings” so we could be confident he wasn’t hungry. With your supply issues, you’ll probably need to pump at the same time, but it’s worth trying.
DH did sometimes sleep in another room when he needed good sleep. I don’t think that, if you’re not ready to CIO, this is a bad suggestion.
Try to ditch the pacifier. I know it’s hard, and seems counter-intuitive if you’re trying to soothe him, but I read somewhere that it can be really disconcerting to them. Like, if you went to bed every night in your room and woke up on the front lawn, that’d freak you out, too, right?
I’d maybe try a swing. It might feel like going backwards to you, but he’s already learned to go to sleep drowsy but awake, so you may find some success here. Maybe when you’d normally give up and co-sleep, transfer him to a swing and see what happens?
It’s also POSSIBLE that your little guy is over-tired. Most babies transition to 2 naps closer to 8 or 9 months, and right now, your little guy should probably not have daytime wake times of more than 2-3 hours. Sleep begets more sleep for babies, so maybe fighting for a 3rd nap however you can make it happen (car, stroller, whatever) might help with bedtime.
When we used CIO at 6 months, we didn’t night wean (I still feed once at nearly 13 months–my guy has weight issues). It was just a way to teach him to fall back asleep. It doesn’t work for everyone, it’s hard work, and it’s not magical, but do keep an open mind to it in the future.
As for the compromise, here’s what I’d do in your position.
I’d pick 4-5 things I’d like to try. Maybe from my list here, maybe things you haven’t tried yet that you’ve read about, etc. I’d agree to try each of those things for 2 weeks. I’d let DH pick the order. Maybe last one is a sleep consultant (I’ve never used one, though I’ve been tempted). If, after these things have been tried (whole-heartedly and consistently), then you try CIO.
Just some thoughts. This is so hard, and I was so right there with you. We still have sleep struggles, but have made so much progress. It sounds like you are doing SO MUCH right, and you should be proud of yourself for that. Some babies are better sleepers (or eaters, or players, or whatever) than others, and having a bad sleeper is tough on everyone. I always tell myself eventually he’ll be a teenager and I won’t be able to get him to STOP sleeping (though that doesn’t help in the middle of the night).
I also belong to an on-line sleep community that totally rocks. It is full of a lot of moms with a lot of baby sleep experience as well as the “expert,” who chimes in a lot. If you’d like to have a link, PM me. They may have a lot of suggestions, too.
One more thing I’d like to ask is what time bedtime is. That might give me other ideas.