Oh yeah, the swaybar bushings feel quite hard (a couple of steps away from something like derelin), but don't look worn out. Plus, I have noticed that one of the tie rod ends is a bit stiff feeling.
 

They are sway bar end links.

"I completely removed the front sway bar, and now there is no noise whatsoever. It also feels a tonne softer to drive in a straight line (which seems odd to me...)."
 
Softer ride and slow turn in with no front bar is not surprising lol.
 
If the noise is gone, it would seem the knock must be associated with the front sway bar.
Check for any slack in the end link joints. If they are not worn they should be stiff to move.
Also check underneath the sway bar brackets for any chassis cracks.
 
You can buy new sway bar bushes but their sizes vary on different models. What year model is your car? 
Anyway you will need to measure the bar diameter and the mounting bracket dimensions because a good fit is important.

BTW assuming you have the stock struts (not coilovers) if you need to replace the front links, here is a good quality set at a reasonable price. I use these myself.
http://www.ebay.com.au/itm/New-Front-Drop-Link-Anti-Roll-Bar-Toyota-mr2-SW20-Delphi-PAIR-/200916068981
 

Thanks for that guys. 
 
Yeah, I'm kind of hesitant to go ahead and bite the bullet on these because I am interested in upgrading the suspension and I am not sure if i will even end up running the suspension in the end, or if i will get shorter sway bar ends with the new shocks...
 
I really like the idea of the BC coilovers and they are just on the cusp of my budget, but I am quite worried at 5&8kgs they're considerably heavier springs than any of the other combinations out there that people use. I am basically after a bit of lowering for aesthetics (I beleive that 1" is fairly standard), a slightly stiffer ride, of course any improvement in handling would be welcomed, but I am really after the lowering and a decent ride for my daily driver.
 
What would you guys suggest? I'm really not looking to pay any more than the BC's cost ($1300) and even that is a bit rich for my taste.

Oh and it's a 95 bathurst.

Anyone.... :/