Back on September 15, 2010 I wrote a little piece, “Frequency – Scrabble and the actual frequency of letter usage in English”, about what seemed to me to be a puzzling mis-match between the number of tiles in Scrabble and my superficial sense of the frequency of occurrence of letters in English.  While I was away in Hong Kong in December, I received the following email note from David T. Wong,

Excuse me for being very late to comment about the piece you wrote last year regarding the frequency of letters used in the English language. The article was posted on September 15, 2010, but I have only recently just read it. Anyway, in the article you had given the assumption that the frequency of letters used in English should be based upon their frequency as occurring in the Concise Oxford Dictionary. I have no objection to using that dictionary for the frequency of letters used in English words, but to assume that the frequency occurring in English words is the same as the frequency occurring is English usage is an absolutely untrue assumption. This is because not all words are used equally.

The simplest way to explain this is that while there is only one word “quiz” in the dictionary, and only one word “the” in the dictionary, why would you expect that the two words (and their component letters) to be used with equal frequency? My point is that you will hear (or see) the word “the” (and its component letters) hundreds of times more frequently than you will see or hear the word “quiz”, yet you only counted each letter (T, H, E, Q, U, I, Z) only once in your analysis of the dictionary. Most analyses of English usage that I see reveal a rather different ranking for letter frequency from the one you showed in your posting of 15 September 2010.

Thank you for your time in considering my opinion on the matter.

In thinking again about this question it struck me that in the case of Scrabble, though a word may be appear more than once on the board, there is no real connection between the words on the board and words used during conversation and in writing. Scrabble players select words to place on the board based on the letters available to them and the maximization of the score based on the scoring rules of the game. One could play Scrabble with just the lexicon of chemistry and find the game works quite well, though with a much smaller universe of likely players.

Thus, the issue for the frequency of the use of the letters in the lexicon of English is the appropriate question, not the frequency of the letters as found in a real stream of communication or “usage”.

This frequency topic reminded me of how peculiar frequency can be in human language.  Consider the English phoneme “th” (e.g. in the word “then”) in words in a lexicon of over 70,646 in an advanced learner’s dictionary and the frequency of use in the spoken text. The phoneme “th” occurs in only 0.12% of the words, yet comprises 3.56% of the phonemes in spoken usage. It ranks 43rd out of 44 phonemes in English as occurrences in the lexicon but 6th in spoken usage.

Below is a chart for further exploration.