The southern exposure is important in the city, and also in any house that wants to take advantage of solar energy, because south is the direction that the sun comes from (assuming that you live in the northern hemisphere). This happens because the earth is a sphere and spins on an axis that is tilted 23 degrees (see this page for information on the tilt). It is easiest to understand why sun comes in your southern-facing windows by looking at how a house sits on the planet and how the sun shines on the house in summer and winter. The two diagrams below help you see what is happening:
The arrangement of the house and sun in the summer
The arrangement of the house and sun in the winter
You can see that, both in summer and winter, the wall of the house that faces the south pole is the one that faces the sun. In these diagrams you can also see that the apparent angle of the sun to the house changes in summer and winter. In the summer the sun is much higher in the sky:
You may have heard that having deep eaves on your house will block the summer sun but let the winter sun in. This change in the sun's angle between summer and winter is why that works.
A good way to visualize this for yourself is to get a globe and tape a Monopoly house, sugar cube or dice to it to represent your house. Then use a flashlight to represent the sun and rotate the globe. Try both summer and winter orientations for the axis. You will be able to see the difference between the angle of the sun in summer and winter, and you will also be able to see how the sun's position changes between morning, noon and evening as it rises and sets.
The following links are interesting: