Addons will increase them memory usage, but even without addons, firefox still has some strange behavior.
Some say it is a memory leak while others say it's not a true memory leak, you can try tweaking some settings in about:config, but there is nothing out there that will "fix" it for everyone:
My firefox 3 was getting ungodly slow, especually if was typing out an address I'd have to wait almost 10 seconds thru as the browser locked up til I could type further.
I've since then switched to Safari 4 and enjoying it. Much much much faster, and is 100% CSS3 compliant.
Right-click your Firefox launch icon, click Properties, click the Compatibility tab, make sure the check box under Compatibility mode is unchecked and restart Firefox.