If you have read How Air Conditioners Work, then you are familiar with the basic process that allows an air conditioner to concentrate cold in one set of coils (inside the house) and heat in another set (outside the house). The article explains the process in great detail.
Imagine that you took an air conditioner and flipped it around so that the hot coils were on the inside and the cold coils were on the outside. Then you would have a heater. It turns out that this heater works extremely well. Rather than burning a fuel, what it is doing is "moving heat."
A heat pump is an air conditioner that contains a valve that lets it switch between "air conditioner" and "heater." When the valve is switched one way, the heat pump acts like an air conditioner, and when it is switched the other way it reverses the flow of Freon and acts like a heater.
Heat pumps can be extremely efficient in their use of energy. But one problem with most heat pumps is that the coils in the outside air collect ice. The heat pump has to melt this ice periodically, so it switches itself back to air conditioner mode to heat up the coils. To avoid pumping cold air into the house in air conditioner mode, the heat pump also lights up burners or electric strip heaters to heat the cold air that the air conditioner is pumping out. Once the ice is melted, the heat pump switches back to heating mode and turns off the burners. (Geothermal heat pumps solve this efficiency problem.)
Here are some interesting links: