When we work with semantic and pragmatic models that traffic in discourse referents, what is it that we are attempting to model? The most common answer is to treat discourse referents as elements of conversational contexts. Contexts are then understood in one of two ways. Some understand them in terms of our extralinguistic psychology—as bodies of shared beliefs or other mental representations. Another approach is to think of contexts as records of conversation that are wholly under grammatical control. I will argue that neither of these views does justice to the phenomena that discourse referents have been posited to understand. Instead, we should think of discourse referents as "object files," which are mental representations that we construct in order to track entities over time, both in perception and in cognition. I will discuss some of the psychological and philosophical literature on object files and theorize about how the our object-file system(s) might interface with the language system. Finally, I will say something about how this project fits into a big-picture understanding of how language and extralinguistic cognition fit together.