Overselling basically means to sell beyond the means of delivery.
It is largely used in low-end hosting industry because it keeps hosting prices at low rates.
And it is also widely abused as many companies go too far with it to keep prices lower than the competition.
Basically, all hosting plan which promise "unlimited" space or "unlimited" bandwith are overselling, simply because does not exist a thing like an "unlimited" hard disk...
Overselling is also common in other industries such as the telecommunication sector.
As the name indicates, it means you sell more resources than there you have really available.
An example: a Reseller Plan which comes with 10 GB of space and 20 GB of bandwidth: you can use it to create domain hosting accounts with very high or even UNLIMITED space and bandwidth allocations — e.g., you could create 100 hosting plans with 1 GB space and 2 GB bandwidth each. Experience shows that 90% of customers do not even use 30 MB, yet many customers want a lot of space and bandwidth, even if they don't need nor use it.
Therefore, rather than filling up only 10% of the space and bandwidth you bought you will be able to utilize your account to maybe 90% and make a much larger profit.

It's obviously a riskly policy: when an user require all the resources you promised, and there're no more, what will you do?