Existing studies suggest that English speakers conceptualize time on both the sagittal and transverse axes (Casasanto & Jasmin, 2012), whereas Mandarin speakers conceptualize time on both the sagittal and vertical axes (Boroditsky 2001; Scott, 1989). It has been suggested that the different temporal directions on the sagittal dimension between the two languages are likely to be caused by the different emphases of temporal sequences: deictic time vs. sequential time. While a large amount of literature has focused on differences across the two languages in terms of using different axes, very little has looked at differences that exist within axes. I report findings from English monolinguals, Mandarin monolinguals and Mandarin-English (ME) bilinguals on an explicit task that involves pointing directions for temporal words. It showed that English monolinguals associated the future with front and up; the overt encoding of metaphor has a significant effect in Mandarin but not in English. More importantly, ME bilinguals showed intermediate patterns. The current study tested cross-linguistic influences on the perception of temporal information. It found that when two languages encode time with different spatial words, both language and spatial cues can affect bilingual speakers’ associations between time and directions. Future studies could test other languages, such as Māori, which see the past as ahead and the future as behind based on visual accessibility.