The easiest & by far the cheapest way is to ask relatives & friends to give them from their daily mail.
Another method would be be exchange your duplicates stamps with other collectors in school & colleges. But ascertain the value of your stamps by checking the prices in catalogues such as Stanley Gibbons or Scott's. Stamps of equal value according to the catalogues may be exchanges directly.
A slightly more expensive way would be to make overseas pen friends & exchange stamps of their countries with yours. Your only cost would be the postage accrued on such exchanges. Resources permitting the ideal way is to buy from your your friendly local stamps dealer, or browse through stamps auctions!
Generally a good way to start is by buying a world packet which would contain different stamps in from different countries. These stamps must be sorted out country wise. While sorting the stamps the collector learns to identify the stamps of different countries. Reading, visiting stamp shows & exhibitions, joining a philatelic society is the key to knowledge of philately. Stamp societies provide a link between the collector & his hobby. These societies also hold seminars, workshops, library facilities & other philatelic services to promote this hobby.
Stamp exhibitions provide opportunities to see famous collections & purchase material from dealers attending these exhibitions & also interact with other collectors.