I read an interview with Warren Buffet shortly after he announced he was giving all that money away to Bill Gates. People accused him of being stingy, he said early on that he realised early on it made more sense to keep the money and grow it than give it away, because he could grow it so much larger.
Even with a pretty average ROI (6%), every $10,000 will turn in to $130,000 in 45 years. Manage a high return (15%) and that $10,000 is over $5 million... in other words, the younger you are the less you should donate to charity.