I ended up having to exclusively pump (so far) since my son was a preemie. Your insurance will get you a pump, but it won’t come with much (take it even if you don’t know you’ll use it, you might and then won’t have to try to get it – the LC in the hospital had them bring one to my room and charge my insurance). I had to get different size flanges, but you should be able to get that stuff from a LC at the hospital before you go home if you need it.
Now I’m back to work so I have to pump here 3x a day. I had to buy:
A bag to carry the pump, etc, in.
A handsfree pumping bra (two, actually, they are indispensible)
A manual pump for $30 for when I’m just out running errands and don’t want to set up my electric.
Bottles for feeding, or at least nipples for the (most likely) Medela bottles
A little cooler and icepack (and more bottles) to carry breastmilk home from work.
A million breastmile storage bags
A bottle brush (we use the regular drying rack)
If you plan to return to work, you’ll need all that stuff. If not, I recommend a manual pump (the Medela ones works great) for when you’re out and about and get engorged, and something to store milk in for if you pump before you leave baby for a while. Bottles are fine if you’re just pumping today for feeding tomorrow, otherwise you’ll want to have some other storage system so you can freeze it. Then even if you feed formula you’ll need bottles. They usually come with infant nipples, so just buy a few. Get one of the more boob-shaped ones if you want to continue to breasfeed, otherwise get whatever. Also get a script for Newman’s Nipple Cream before you leave the hospital.
Even when you return to work, you can breastfeed at night and then choose whether or not you want to pump during the day. It doesn’t have to be all or nothing. It is a drag to do it though, won’t lie.