When you buy shares, you pay the full amount according to number of the share and price of the share. But if you pay only a part of the amount, thats called buying on margin.
For example, if you buy a 1000 shares worth GBP 1 per share, you pay GBP 1000 and buy the shares. But when you choose to buy shares in margin, you will be required to pay only for 50% of shares, which is GBP 500. The remaining amount will be paid by the brokerage firm.
But there are limitations with regards to the margin depending upon the stock brokerage firms. Some brokerage firms set the minimum limit in the form of amount say GBP 2000, which means you have to pay atleast GBP 2000 to buy the shares or they might set the limit in the form of a certain percentage which means you have to pay the percentage instead of the fixed amount. But the catch is you have to pay the interest for the amount they are giving you. In the mean time they will be holding your a certain amount of your shares.
If you are unable to pay the interest, your shares will be sold by your brokerage firm and the amount will be recovered. This is something like buying a home in mortgage. When you are unable to pay the mortgage, your home is seized. Just like that.
When you are buying shares in margin, you have to make sure that your share value is rising to a level, so that you can make profit and pay interest as well. But as per law there is a limit upto which you can hold the shares in loss. But if the share value goes below that limit, then your shares will be sold and amount you owe to brokerage firm will be recovered. In this process the risk is mainly for the investors and not for the stock brokers.
Buying on margin has huge risk as mentioned above. You might end up losing your entire capital, as you are not in complete control of your stocks. So buying shares in margin is not really advisable for the beginners of stock trading as they dont know the behaviour of the shares. If you have reached a level where you can judge the movement of shares, then you can start buying shares in margin.