I am currently in the Robson Media Centre, underneath Robson Square (photo) in Vancouver. Robson Square itself is a fantastic place, there are even things to do above and below it! During the Olympic Games, you can go and skate on the ice rink below Robson Square, take a ride on the zipline overhead Robson Square. If those are not for you, there are always musicians performing on the two stages right beside the ice rink.
The Vancouver Art Gallery (photo) is just north of the rink and is a good place to get away from the excitement and the noise. Also, the UBC bookstore is open. For all of these events except a long wait, especially for the zipline. You might wait up to 6 hours to get a ride!