Humans are very good at judging things. In fact, some people in your world are paid very well to do it! You have limited sensory criteria to consider when it comes to putting something in a category.
Your physical world has a multitude of events and situations occurring on many levels. Your five senses pick up a very small percentage of them. When you go out on your street, you focus on a very narrow range of events.