  Finding claims that researchers have made considerable progress in artificial
intelligence over the last several decades is easy. However, our everyday
interactions with cognitive systems (e.g., Siri, Alexa, DALL-E) quickly move
from intriguing to frustrating. One cause of those frustrations rests in a
mismatch between the expectations we have due to our inherent,
folk-psychological theories and the real limitations we experience with
existing computer programs. The software does not understand that people have
goals, beliefs about how to achieve those goals, and intentions to act
accordingly. One way to align cognitive systems with our expectations is to
imbue them with mental states that mirror those we use to predict and explain
human behavior. This paper discusses these concerns and illustrates the
challenge of following this route by analyzing the mental state 'intention.'
That analysis is joined with high-level methodological suggestions that support
progress in this endeavor.
