Although it is possible to get creative with how you use them, for understanding I'll explain...
Sites
Imagine you have small business depots all around the country, each one has a small inventory holding, a small warehouse. Each one of these would be classed as a site, and you can give it a site ID.
For example, you could choose the name of the Site ID to be the town,
Site ID
ANNARBOR
NEWAYGO
FLEET
LANCING
(in reality you might want a naming scheme that allows expansion with more than one per town!).
Site are a logical segment in GP, so many reports and operations can be performed by Site, for example stock counts. This makes it easier for that local depot to manage the local inventory.
Bin
Now within each of those sites, say in Boston, you need to know which physical rack that item is held on, often warehouses will be broken up into aisles, bays and shelves within a code, for example A13 would mean aisle A, Bay 1, third shelf.
This location is your Bin, the location holding the stock for that site
As you scroll through the sites in GP, you will see the that you enter a different Bin for each site, this is because the stock may not be held in the same bin at all the different depots around the country.
As I said Sites are a logical segment, so for example, different sites could use a different SOP order type, where each order type is localised to default the Site ID to the local site ID for the depot.
To confuse a little, but I'll risk it, you can also set up dummy sites, so often you will see a site that is only a logical site not a physical one, such as a site called quarantine where stock can be transferred when you wish to prevent it being sold by the sales order entry users (unless they explicitly change the site they want to sell from).
Hope this is enough to draw a mental picture of how it is supposed to be used.
Tim.