Yes, the intake manifold vacuum goes up and down depending on engine load, but there is a one way vacuum check valve that always retains vacuum in the booster circuit. Without it the brake booster wouldn't work.
Early boosters had the check valve mounted on the booster in the frunk, and later boosters had it back on the intake manifold.
Would be easy to modify the early system with an extra check valve in the engine bay to have a reliable vacuum supply there.