Intelligent robots of the future will need to acquire an understanding of the situation in which they are as well as make predictions about what could happen next. To accomplish this, they will need to understand the events that are happening in front of them or events in which they participate. I present the theory of cognitive primitives which provides a compositional system for modeling events and actions. Experimental results using humanoid robots answering questions about events at a table top (where robot action may be required), will also be presented.