Three experiments were conducted to investigate how we recognize the space surrounding our bodies, especially the space behind us. Undergraduates performed spatial perspective taking tasks after having learned the locations of five objects on a table while blindfolded. Results showed that (1) discrepancy between imagined orientation taken in the task and actual body orientation at that time had an effect on accuracy of subject's performance, (2) discrepancy between imagined orientation taken in the task and body orientation when they had learned the locations of objects had an effect on speed of subject's performance, (3) the complexity of object-array structure influenced whether the effect (2) emerged or not. Based on Easton & Sholl (1995)'s model of spatial cognition with some supplementary assumptions, these results were interpreted in terms of the interference between a self-reference system at sensorimotor level and a self-reference system at representational level superposed on a representation of object-to-object relations, and the interference between a self-reference system involved in a “subject-to-object-object” representation and a self-reference system at representational level.