Oxygen sensor wouldn't be having any effect in the case you're describing as it really only comes into play in fairly steady state 'cruise' conditions. The rest of the time the ECU is just running straight from a set of maps. The changed fuel pump does make it look more like a fuel pressure issue. Unfortunately I don't have a gauge so have no easy way to check. However, you could inspect the pressure regulator (small mushroom shaped device with a vacuum line and a couple of fuel lines in and out) which I think is located near the passenger side end of the fuel rail (it may be at the other end). The thing to check is whether the vacuum line is connected or not and whether it is open or blocked.
The regulator is designed to increase fuel pressure when boost is applied through the vacuum line. It is possible that the regulator is stuck or has failed in a high pressure position or that the new fuel pump is flowing more fuel than the regulator can handle.
I'm offline (apart from phone) so can't post a pic of the diagnostic box location, but its pretty obvious at the rear of the engine bay on the drivers side