Lexical diversity


Lexical diversity is one aspect of 'lexical richness' and refers to the ratio of different unique word stems to the total number of words. The term is used in applied linguistics and is quantitatively calculated using numerous different measures including Text-Type Ratio, vocd, and the measure of textual lexical diversity.
A common problem with lexical diversity measures, especially TTR, is that text samples containing large number of tokens give lower values for TTR since it is often necessary for the writer or speaker to re-use several function words. One consequence of this is that lexical diversity is better used for comparing texts of equal length. Newer measures of lexical diversity attempt to account for sensitivity to text length.

Definitions

In a 2013 article Scott Jarvis proposed that lexical diversity, similar to diversity in ecology, is a perceptual phenomenon. Lexical redundancy is a positive counterpart of lexical diversity in the same way as lexical variability is the mirror image of repetition. According to Jarvis's model, lexical diversity includes variability, volume, evenness, rarity, dispersion and disparity.
According to Jarvis, the six properties of lexical diversity should be measured by the following indices.
PropertyMeasure
VariabilityMeasure of Textual Lexical Diversity
VolumeTotal number of words in the text
EvennessStandard deviation of tokens per type
RarityMean BNC rank
DispersionMean distance between tokens of type
DisparityMean number of words per sense or Latent Semantic Analysis