The biggest problem (IMO) from my observations is that when people convert to ITB's, they do not maintain sufficient length of intake runner. What this does is requires you to use higher RPM to get the same wave order as the stock intake or you end up using a lower wave order at the same RPM, which has less power potential (anything after the 5th wave is pretty marginal).
Example. The 4AGE 16V Bigport has runner lengths around ~450mm long before reaching the main plenum chamber, but when fitting ITB's with an adaptor, that is easily brought back to ~300mm or less. With a 480mm runner length, 7000RPM is approx the 4th wave order. If the intake runner is shortened to 320mm during the ITB conversion, this puts it on the 6th wave order at 7000rpm and you lose the performance, hence why you need to rev it harder... On the flip side, if you can get the runner to 600mm, you can get it onto the 3rd wave order at 7000rpm but you will also pick up the 5th wave order at 4200rpm.
Toyota did similar things with their ACIS intake design to ensure that they could have good low down torque by having long runners through a divorced chamber with a butterfly that would open up later on in the RPM, essentially shortening the runner length.

This is found on the 3VZ and 1MZ motors as well.
The other thing to consider is, just because your going to ITB's doesn't mean you have to go to an open air velocity stack design, there is plenty of performance available with a plenum design (F1 is an example).