“The text from Augustine’s Confessions with which Wittgenstein opens the Investigations registers a strong sense of how the self-transparent little soul looks out from its head, hears the adults making various noises, watches them (through its eyes) as they lumber towards some item of middle-sized dry goods, and then suddenly, on its own, makes the connection, in its own mind, between the sounds the adults emit and the objects that they touch.  Augustine pictures his infant self as already aware of its identity (what is going on inside its own mind) and of what is going on around it (outside its mind), prior to and independently of its mastering the arts of speech.  The text offers ‘a particular picture of the essence of human language’ (PI 1).  It is important to notice, however, from the outset, that the ‘words name objects’ doctrine of language which Wittgenstein at once extracts from the text is interwoven with the idea that meaning is always in the head: the last remark in the Investigations has to be allowed to illuminate the first one.  As Waismann wrote, recapitulating the idea:

 What we object to is the idea of the contents of different people’s minds as shut off from each other by insurmountable barriers, so that what is experienced is eternally private and inexpressable – the idea that we are, so to speak, imprisoned behind bars through which only words can escape, as though it were a defect in language that it consists wholly of words (Waismann, Principles, p. 248).

Wittgenstein thinks, we badly need the reminder.  Indeed, the only problem that he has with Augustine’s story is that what is presented as secondary and marginal to self-understanding needs to be acknowledged as fundamental.  He only wants to draw attention to what Augustine’s picture leaves in the background” (Fergus Kerr, Theology after Wittgenstein, p. 56, 57).