It has been pointed out that gaze alternation by infants, which is the behavior of alternately gazing at a caregiver and at particular objects, is related to the process of the development of intentional agency. Intentional agency is defined as acting with a desired goal and means. In this paper, we adopt a theoretical hypothesis that the infants understand others' intentions based on intentional agency, and we consider how to construct a computational model of intentional agency. We designed a model of an infant agent which acquires gaze alternation through interaction with caregivers based on a reflex behavior and an emotional behavior. First, the agent learns the visual orientation of gazing at a target in the center of the visual field as the reflex behavior. Based on the visual orientation, the agent learns to gaze in the same direction as the caregiver's focus. The learning is implemented with an association module which is serially connected with the visual orientation module. In the model, the agent associates the caregiver's focus with an object, and orients the agent's eye to gaze at the object. This behavior uses visual orientation as a means to attain the agent's goal of gazing at the same object. The internal states composed of goals and means are considered to be intentional agency. Second, we add two emotional states, ease and anxiety, to relate an emotional behavior to the serial architecture acquiring intentional agency. The agent looks back at the caregiver when the agent comes to the anxiety state. The emotional behavior provides opportunities of interaction with caregivers to the infant agent. Finally, we discuss how intentional agency functions as a basis of understanding others' intentions. Through this discussion, we propose that a nested structure of intentional agency between self and other is a primitive mechanism of understanding others' intentions and shared intentionality.