Recent work in natural language processing has shown that large language models, which are trained purely to “fill in the blanks” in natural text, can learn semantic phenomena like entailment relations between sentences to some degree. But are there fundamental limits on what can be learned about a language’s semantics just by observing the distribution of texts that speakers produce in that language? I will discuss preliminary theoretical results on this question. Specifically, I will prove that it is straightforward for a distributional learner to resolve entailments if its training data is produced by truthful speakers whose productions are independent conditioned on the world state. Notably, these assumptions are satisfied in a basic rational speech acts model. It is an open question how these results change as these assumptions are relaxed to be more realistic, which I hope to analyze in ongoing work.