You don't rob somebody in the middle of the day right next to the gates of the city. The only reason nobody is doing something is because no one is THERE because it isn't peak hours. But RP wise, there would be guards on the walls, people on the street, all sorts of things.