@Opus: Oooohhh, good point on the Olympics; I forgot that was next summer!
OP, I really think it depends on what “level” of vacation you’re looking for. You can spend 4k or less doing it cheap, but you can also spend quite a bit more without really trying, especially in Paris and London. Those are just expensive cities to be in.
I’m trying to remember what we budgeted for our honeymoon a couple years ago. We went kinda “mid-line,” I’d say, cutting in some areas and splurging in others. I think we spent about $150/day on food (that was roughly 50 euros/person/day); we usually just had coffee and pastries for breakfast, street food for lunch (since we were out walking), grocery store snacks in between meals instead of buying from a cafe, and a mid-line cafe for dinner. We also don’t drink much, so we spent almost no money on alcohol. We had a couple dinners from bakeries and one splurge on eating at the Eiffel Tower, so it all evened out.
For transportation, at least while we were in the cities, we spent practically nothing and walked everywhere. We specifically chose hotels near the city center, so we didn’t have to use the metro or taxis (except to and from the airport/train station). However, both London and Paris have tour buses to all the major tourist sights (RedBus in Paris). Cheap to buy day passes for, and you can hop on/off as you like. Also, the Eurostar train is cheap for traveling from city to city, and really fast, too. For sightseeing, we decided ahead of time which sights we really wanted to buy passes for, though we walked by a lot of other stuff. Since you have a limited time, it’s really impractical to buy tickets to everything, narrow it down to maybe one or two a day; it saves money and you’ll get more time at those sights to actually enjoy it. Also, look into city passes, which are a higher upfront cost, but give you big savings on the popular tourist sights. I think our total travel and sightseeing budget was around $5k (but we also spent 14 days in Europe and traveled to 5 cities instead of three, plus we had to rent a car at one point, which was a big expense).
Lastly, hotels. It’s tricky because you may spend a little more on hotels to save on travel, and you want to stay in a safe neighborhood, which can jack the price up. I think, generally, we tried to stay in the range of $150/night, and generally stayed in small, clean hotels in decent areas of each city.