Cross-linguistic studies in several semantic domains, such as kinship, color, and numeral systems, suggest that word meanings are adapted for efficient communication (see Kemp2018 for a review). However, until recently it had remained largely unknown to what extent this proposal can account for soft semantic categories and inconsistent naming, that could appear to pose a challenge to the notion of efficiency, and how pressure for efficiency may relate to language evolution. Recently Zaslavsky . (2018; henceforth ZKRT) addressed these open questions by grounding the notion of efficiency in a general information-theoretic principle, the Information Bottleneck (IB; Tishby1999). ZKRT tested this formal approach in the domain of color naming and showed that the IB principle: (1) accounts to a large extent for cross-language variation in color naming; (2) provides a theoretical explanation for why observed patterns of inconsistent naming and soft semantic categories may be efficient; and (3) suggests a possible evolutionary process that roughly recapitulates Berlin and Kay’s (1969) discrete implicational hierarchy while also accounting for continuous aspects of color category evolution. However, it is not yet clear to what extent these results may generalize to other semantic domains, especially those that are fundamentally unlike color.
Here we test the generality of this theoretical account by considering two additional semantic domains: artifacts and animals. These domains are of particular interest in this context because they are qualitatively different from color, they have not previously been comprehensively addressed in terms of efficient communication, and at the same time it is possible to apply to them the same communication model that has previously been used to account for color naming.
First, we consider naming patterns for household containers. This is a semantic domain in which categories are known to overlap and generate inconsistent naming patterns Ameel . (2005, 2009). Although it has previously been shown that container naming in English, Spanish, and Chinese is efficient compared to a large set of hypothetical naming systems Xu . (2016)
, that demonstration did not consider the full probability distribution of names produced by different speakers, did not explicitly contrast monolingual and bilingual speakers, and was based on a smaller set of stimuli than we consider here. In this work we show that the full container-naming distribution in Dutch and French, including overlapping and inconsistent naming patterns, across a large set of stimuli, both in monolinguals and bilinguals, is near-optimally efficient in the IB sense.
Second, we test the evolutionary account of ZKRT in the case of animal categories. By analogy with Berlin and Kay’s implicational hierarchy of color terms, Brown1984 proposed an implicational hierarchy for the evolution of animal taxonomies based on cross-language comparison. We show that aspects of this hierarchy are captured by a sequence of efficient animal-naming systems along the IB theoretical limit. Our results also support the view that both perceptual and functional features shape animal categories across languages Malt (1995); Kemp . (2018).
The remainder of this paper proceeds as follows. First, we review the theoretical framework and formal predictions on which we build. We then present two studies that apply this approach to the aforementioned semantic domains.
2 Theoretical framework and predictions
We consider here the theoretical framework proposed by ZKRT, which is based on a simplified interaction between a speaker and a listener (Figure 1), formulated in terms of Shannon’s 1948 communication model. The speaker communicates a meaning , sampled from , by encoding it into a word , generated from a naming (or encoder) distribution . The listener then tries to reconstruct from the speaker’s intended meaning. We denote the reconstruction by , and assume it is obtained by a Bayesian listener.111The reconstruction of a Bayesian listener with respect to a given naming distribution is defined by . These meanings, and , are taken to be mental representations of the environment, defined by distributions over a set of relevant features. For example, if communication is about colors, then may be grounded in a perceptual color space, and each color would be mentally represented as a distribution over this space.
Under these assumptions, efficient communication systems are those naming distributions that optimize the Information Bottleneck (IB; Tishby1999) tradeoff between the complexity and accuracy of the lexicon. Formally, complexity is measured by the mutual information between meanings and words, i.e.:
which roughly corresponds to the number of bits used to encode meanings into words. Accuracy is inversely related to the discrepancy between and , measured by the expected Kullback–Leibler (KL) divergence between them:
Achieving maximal accuracy may require a highly complex system, while minimizing complexity will result in a non-informative system. Efficient systems are thus pressured to balance these two competing goals by minimizing the IB objective function,
where controls the efficiency tradeoff. The optimal systems, , achieve the minimal value of equation (3) given , denoted by , and evolve as gradually shifts from to . Along this trajectory they become more fine-grained and complex, while attaining the maximal achievable accuracy for their level of complexity. This set of optimal systems defines the theoretical limit of efficiency (see Figure 2).
If languages are pressured to be efficient in the IB sense, then for a given language with naming system , two predictions are made. (1) Deviation from optimality, or inefficiency, should be small. This is measured by , where
is estimated such thatis minimized. (2) The dissimilarity between and the corresponding IB system, , should be small. This is evaluated by a dissimilarity measure (gNID) proposed by ZKRT. In addition, ZKRT suggested that languages evolve along a trajectory that is pressured to remain near the theoretical limit.
These predictions were previously supported by evidence from the domain of color naming. To apply this approach to other domains, i.e. to instantiate the general communication model, two components must be specified: a meaning space, which is the set of meanings the speaker may communicate; and a prior, , also referred to as a need distribution Regier . (2015), since it determines the frequency with which each meaning needs to be communicated. In the following sections we present two studies that follow this approach and test its predictions in qualitatively different semantic domains.
3 Study I: Container names
The goal of this experiment is to test the theoretical predictions derived from IB in the case of container naming. It is not clear whether previous findings for color would generalize to this case for several reasons. First, the representation of artifacts is likely to involve more than just a few basic perceptual features, unlike color. Second, categories in this domain are believed to be strongly shaped by adaptation to changes in the environment Malt . (1999). At the same time, container categories tend to overlap, as in the case of color categories, posing a similar theoretical challenge to explain this observation in terms of communicative efficiency. Finally, the bilingual lexicon in this domain has been extensively studied, and it has been shown that bilingual naming patterns tend to converge Ameel . (2005, 2009). However, it is not yet clear whether this convergence, or compromise, comes at a cost in communicative efficiency, or whether it may actually be formalized and explained in terms of efficiency.
To address these open questions, we consider sorting and naming data collected by White2017, relative to a stimulus set of 192 images of household containers (see Figure 3A for examples). This set is substantially larger than those used in previous container-naming studies (e.g. Malt1999,Ameel2005), thus providing a better representation of this semantic domain. In the naming task, 32 Dutch and 30 French monolingual speakers, as well as 30 bilingual speakers, were asked to provide names for the containers in the stimulus set. Bilingual participants performed the task once in each language. The container-naming distribution in each of the four conditions (language linguistic status) is defined by the proportion of participants in that condition that used the word to describe a container . A separate sorting task was performed by 65 Dutch speakers, who were asked to organize all containers into piles based on their overall qualities. Participants were also allowed to form higher-level clusters by grouping piles together. White2017 evaluated the similarity between two containers, denoted here by , based on the number of participants that placed them in the same pile or cluster (see White2017 for detail). In both tasks, participants were instructed not to take into account the content of the object (e.g., water).
We ground the meaning space in the similarity data, following a related approach proposed by Regier2015 and Xu2016. While these data are from Dutch speakers, there are only minor differences in perceived similarities among speakers of different languages Ameel . (2005). Therefore, we assume that these similarity judgments reflect a shared underlying perceptual representation of this domain. We take to be the set of containers in the stimulus set, and define the mental representation of each container by the similarity-based distribution it induces over the domain, , where
is taken to be the empirical standard deviation of. In contrast with the case of color, in which these mental representations were grounded in a standard perceptual space, here there is no standard perceptual space for containers, and so our assumed underlying perceptual representation requires further validation, which we leave for future work. We define the need distribution, , by averaging together the least informative (LI) priors for the different languages, as proposed by ZKRT. We used only the monolingual data for this purpose, and regularized the resulting prior by adding to it and renormalizing.
We estimated the theoretical limit of efficiency for container naming by applying the IB method Tishby . (1999), as ZKRT did in the case of color naming, here with values of . We evaluated the empirical complexity and accuracy in the four naming conditions by entering the corresponding naming distributions in the equations for and . The results are shown in Figure 2 and Table 1. It can be seen that container naming in Dutch and French lie near theoretical limit, both for monolinguals and bilinguals, and that bilinguals achieve similar levels of efficiency as monolinguals (Table 1). In all four cases, the corresponding IB solution is at , suggesting that there is only a weak preference for accuracy over complexity in this domain, as also found for color naming.
Consistent with the empirical observations of convergence in the bilingual lexicon, the complexity-accuracy tradeoffs in bilinguals are closer to each other (Figure 2, orange and red dots) compared to the monolingual tradeoffs (Figure 2, blue and green dots). This may be explained by a need to reduce the complexity of maintaining two naming systems simultaneously, while achieving monolingual-like levels of efficiency in each language. To test this possibility, we compared two joint French-Dutch systems that bilinguals may employ: one that randomly selects one of the two monolingual systems to name objects, and another that randomly selects one of the two bilingual systems. We found a reduction in the complexity of the joint bilingual system compared to the joint monolingual system. Although this is a small effect, it may accumulate across domains to have a substantial impact. In addition, our simple calculation did not take into account similar word forms, which may also reduce complexity Ameel . (2005). Thus, this finding suggests that the convergence in the bilingual lexicon may be shaped, at least in part, by pressure for efficiency.
The remainder of our analysis focuses on the monolingual systems, as they are more distinct and presumably more representative of each language. To get a precise sense of how challenging it may be to reach the observed levels of efficiency, we compared the actual naming systems to a set of hypothetical systems that preserve some of their statistical structure. This set was constructed by fixing the conditional distributions of words, while shifting how they are used by applying a random permutation of the containers. For each language we constructed such hypothetical systems. Table 1 shows that these hypothetical systems are substantially less efficient than the actual systems, and are also less similar to the IB systems. In fact, both languages achieve better (lower) scores than all of their hypothetical variants, providing a precise sense in which they are near-optimal according to IB. One possible concern is that this outcome may be a result of the LI prior, which was fitted to the naming data. To address this, we repeated this analysis with a uniform need distribution. The results in that case are similar (not shown), although as expected the fit to the actual systems is not as good compared to the LI prior.
The low dissimilarity scores for the actual languages, shown in Table 1, suggest that the observed soft category structure in this domain may also be accounted for by the IB systems. This is indeed supported by a fine-grained comparison between the naming distribution in both languages and their corresponding IB systems. To see this, we embedded the 192 containers in a 2-dimensional space by applying non-metric multidimensional scaling (nMDS) with respect to the similarity data, similar to Ameel2009. This was done using the scikit-learn package in Python. We initialized the nMDS procedure with a solution for the standard metric MDS that achieved the best fit to the similarity data out of 50 solutions generated with random initial conditions. For visualization purposes, we assigned a unique color to each container. The resulting 2D embedding and color coding of the containers stimulus set are shown in Figure 3A.
|hypothetical||0.29 (0.02)||0.59 (0.05)|
|hypothetical||0.31 (0.01)||0.56 (0.06)|
The monolingual systems in Dutch and French are shown in Figure 3B, together with their corresponding IB systems. These two IB systems are very similar, although not identical, which is not surprising given that the naming patterns in Dutch and French are fairly similar. Both the actual systems and the IB systems exhibit soft category structure and similar patters of inconsistent naming, as shown by the white dots. In addition, since each category is colored according to its centroid, similarity between the category colors together with their spatial distribution reflect the similarity between the full naming distributions. For example, the IB systems have a category that is similar to fles and bouteille, as well as a category that is similar to doos and boîte in Dutch and French respectively, although these categories in the IB systems are a bit narrower. The IB systems also capture the category tube quite well in both languages. However, there are also some apparent discrepancies. For example, the distinction between bouteille and flacon in French is reflected in both IB systems, although Dutch does not have the same pattern in this case Ameel . (2005).
This analysis shows that efficiency constraints may to a substantial extent explain the container-naming distribution in Dutch and French, including soft category boundaries and inconsistent naming observed empirically, both in monolinguals and bilinguals. It thus supports the hypothesis that a drive for information-theoretic efficiency shapes word meanings across languages and across semantic domains. However, since this analysis is based only on two closely related languages, we were not able to test how well the results for this domain generalize across languages. Important directions for future research include testing whether these results generalize to other, preferably unrelated, languages, and further testing the extent to which the convergence in the bilingual lexicon is influenced by pressure for efficiency. The next section focuses on another semantic domain for which we are able to obtain broader cross-linguistic evidence.
4 Study II: Folk biology
Cross-language variation and universal patterns in animal taxonomies have been extensively documented and studied Berlin (1992), however this domain has not yet been approached in terms of efficient communication. By analogy with Berlin and Kay’s theory, Brown1984 proposed an implicational hierarchy for animal terms, based on data from 144 languages. Brown identified six stages for animal taxonomies, as illustrated in Figure 4A. Languages at the first stage do not have any lexical representation for life-forms. Languages at stages 2-4 add terms for fish, bird and snake, but Brown does not argue for any particular order for these categories. Terms for mammal and wug (“worm-bug”, referring in addition to small insects) are added in stages 5 and 6, again with no implied order. Much of the data analyzed in this domain is not fine-grained, and Brown’s proposal has been criticized Randall Hunn (1984) mainly due to lack of sufficiently accurate data. Nonetheless, his observations can be considered as a rough approximation of cross-linguistic tendencies in this semantic domain. Therefore, in this work we aim at testing whether broad cross-linguistic patterns, as summarized by Brown’s proposal, can be accounted for in terms of pressure for efficiency. More specifically, our goal is to derive from the IB principle a trajectory of efficient animal-naming systems, analogous to ZKRT’s trajectory for color, and to compare this trajectory to the naming patterns reported by Brown. However, unlike previous comparisons to IB optima, due to the nature of available data, here we only attempt to make coarse comparisons.
To derive a trajectory of efficient animal-naming systems, we first need to specify the communication model in this domain. We ground the representations of animals in high-level, human-generated features. Specifically, we consider the Leuven Natural Concept Database De Deyne . (2008), which contains feature data and familiarity ratings for animal classes (e.g., “cat”, “chicken”, etc.). These data were collected from Dutch speakers, and then translated to English. We follow Kemp2010, who considered 113 animal classes and 757 features from this database, and for each feature and class estimated the conditional probability based on the number of participants who generated this feature for that class (see Figure 4B for examples). We take to be the set of animal features, and assume each animal class is mentally represented by the distribution it induces over features, i.e. , as estimated by Kemp2010. In addition, we follow Kemp2010 in using a familiarity-based prior over animal classes, in which the probability of a class is proportional to its familiarity score. We define the need distribution to be this prior.
Given these components, we estimated the theoretical limit for animal naming (Figure 4C) using the same method as before, this time with values of . We then selected the most informative systems with categories. The number of categories, , was determined by considering categories with probability mass . These systems are shown in Figure 4D, where each layer of the hierarchy corresponds to a system and each box corresponds to a category within that system. The top layer, with a single category, corresponds to a non-informative system that does not distinguish between different animal classes. This can be considered as a stage 1 system in Brown’s sequence. The second layer (shown in orange) roughly corresponds to a stage 2 system. It consists of a fish category, as can be inferred from the distribution it induces over features and animals, and another category for all other animals. It lies very close to the origin in Figure 4C, as it maintains little information about most animals. The third layer (shown in red) corresponds to a system with categories for fish and wug, as well as a category that is dominated by birds and mammals. The bird-mammal category has greater probability mass () than the wug category (), suggesting that it is more prominent even though these two categories appear together. This transition deviates from Brown’s sequence in the early appearance of wug (although not strongly weighted here), and in lacking a snake category (although animals from that category do appear in the Leuven database). One possible explanation for this deviation is that the feature data on which we relied were obtained from Dutch participants, and are thus strongly biased toward Western societies. In the next layer (shown in blue), the 3-category system evolved to a 4-category system by refining the bird-mammal category, resulting in a system that roughly corresponds to a Brown stage 6 system, with the exception of snake.
These results suggest that animal naming systems may evolve under efficiency pressure much as color appears to, despite the qualitative difference between these domains. However, in order to test this proposal more comprehensively, fine-grained cross-linguistic animal naming data is required, comparable to the naming data for colors and containers. The fact that systems along the theoretical limit capture some cross-linguistic tendencies in animal taxonomies is notable, given that our characterization of the domain, in terms of features, was necessarily strongly biased toward animal representations in Western societies. This finding supports the idea that to some extent at least there is a shared underlying representation of animals across cultures Mayr (1969), while also raising the interesting possibility of some cross-language and cross-cultural differences in underlying representations. It is also worth noting that the salient features in the IB systems tend to be both perceptual (e.g., “is big”) and functional (e.g., “is edible”), suggesting that both types of features may shape animal categories across languages, and that this may be consistent with pressure for efficiency Kemp . (2018).
Although we introduced the hierarchy in Figure 4D as an account of category structure across languages, the same hierarchy could potentially serve as a model of hierarchical structure within a single language. This within-language interpretation resembles previous applications of the IB principle to language Pereira . (1993), although these applications were based on corpus statistics. The within-language interpretation seems useful in the case of animal taxonomies, a semantic domain with strong hierarchical structure, as opposed to containers and even colors. A possible, yet speculative, reconciliation of the within-language and cross-language interpretations is that speakers may internally represent a hierarchy induced by an evolutionary sequence. For example, Boster1986 showed that English speakers can recapitulate Berlin and Kay’s implicational color hierarchy in a sequential pile-sorting task. Thus, it seems at least possible that a similar phenomenon may also hold for animal categories.
5 General discussion
Artifacts, animals, and colors are qualitatively different elements of human experience, yet our findings suggest that their semantic representations across languages is governed by the same general information-theoretic principle: efficient coding of meanings into words, as defined by the IB principle. We have shown that this theoretical account, which was previously tested only in the domain of color naming (ZKRT), generalizes to container names and animal taxonomies. This finding resonates with the proposal that word meanings may be shaped by pressure for efficient communication Kemp . (2018). However, it goes beyond that proposal by explaining how pressure for efficiency may account for soft categories and inconsistent naming, both in monolinguals and bilinguals, and how it may relate to language evolution.
An important direction for future research is to test to what extent our results extend to other semantic domains, and ideally, to the lexicon as a whole. While it may not be possible to apply this approach to every aspect of the lexicon, we believe that the theoretical formulation considered here may be broadly applicable across semantic domains.
We thank Anne White, Gert Storms, and Barbara Malt for making their container naming and sorting data publicly available. The animal features and familiarity data we used were preprocessed by Kemp2010. We thank Simon De Deyne for initially sharing these data, and for useful discussions. This study was partially supported by the Gatsby Charitable Foundation (N.Z. and N.T.), and by the Defense Threat Reduction Agency (N.Z. and T.R.); the content of the study does not necessarily reflect the position or policy of the U.S. government, and no official endorsement should be inferred.
- Ameel . (2009) Ameel2009Ameel, E., Malt, BC., Storms, G. Assche, FV. 2009. Semantic convergence in the bilingual lexicon Semantic convergence in the bilingual lexicon. Journal of Memory and Language602270–290.
- Ameel . (2005) Ameel2005Ameel, E., Storms, G., Malt, BC. Sloman, SA. 2005. How bilinguals solve the naming problem How bilinguals solve the naming problem. Journal of Memory and Language53160–80.
- Berlin (1992) Berlin1992Berlin, B. 1992. Ethnobiological Classification: Principles of Categorization of Plants and Animals in Traditional Societies Ethnobiological classification: Principles of categorization of plants and animals in traditional societies. Princeton University Press.
- Berlin Kay (1969) Berlin1969Berlin, B. Kay, P. 1969. Basic Color Terms: Their Universality and Evolution Basic color terms: Their universality and evolution. Berkeley and Los AngelesUniversity of California Press.
- Boster (1986) Boster1986Boster, J. 1986. Can Individuals Recapitulate the Evolutionary Development of Color Lexicons? Can individuals recapitulate the evolutionary development of color lexicons? Ethnology25161–74.
- Brown (1984) Brown1984Brown, CH. 1984. Language and living things: Uniformities in folk classification and naming Language and living things: Uniformities in folk classification and naming. Rutgers University Press.
- De Deyne . (2008) DeDeyne2008De Deyne, S., Verheyen, S., Ameel, E., Vanpaemel, W., Dry, MJ., Voorspoels, W. Storms, G. 2008. Exemplar by feature applicability matrices and other Dutch normative data for semantic concepts Exemplar by feature applicability matrices and other Dutch normative data for semantic concepts. Behavior Research Methods4041030–1048.
- Kemp . (2010) Kemp2010Kemp, C., Chang, KK. Lombardi, L. 2010. Category and feature identification Category and feature identification. Acta Psychologica1333216–233.
- Kemp . (2018) Kemp2018Kemp, C., Xu, Y. Regier, T. 2018. Semantic Typology and Efficient Communication Semantic typology and efficient communication. Annual Review of Linguistics41.
- Malt (1995) Malt1995Malt, BC. 1995. Category Coherence in Cross-Cultural Perspective Category coherence in cross-cultural perspective. Cognitive Psychology29285–148.
- Malt . (1999) Malt1999Malt, BC., Sloman, SA., Gennari, S., Shi, M. Wang, Y. 1999. Knowing versus Naming: Similarity and the Linguistic Categorization of Artifacts Knowing versus naming: Similarity and the linguistic categorization of artifacts. Journal of Memory and Language402230 - 262.
- Mayr (1969) Mayr1969Mayr, E. 1969. The biological meaning of species The biological meaning of species. Biological Journal of the Linnean Society13311-320.
- Pereira . (1993) Pereira1993Pereira, F., Tishby, N. Lee, L. 1993. Distributional clustering of English words Distributional clustering of English words. Proceedings of the 31st Annual Meeting of the Association for Computational Linguistics Proceedings of the 31st annual meeting of the Association for Computational Linguistics ( 183–190).
- Randall Hunn (1984) Randall1984Randall, RA. Hunn, ES. 1984. Do life-forms evolve or do uses for life? Some doubts about Brown’s universals hypotheses Do life-forms evolve or do uses for life? Some doubts about Brown’s universals hypotheses. American Ethnologist112329-349.
- Regier . (2015) Regier2015Regier, T., Kemp, C. Kay, P. 2015. Word meanings across languages support efficient communication Word meanings across languages support efficient communication. B. MacWhinney W. O’Grady (), The Handbook of Language Emergence The handbook of language emergence ( 237–263). Hoboken, NJWiley-Blackwell.
- Shannon (1948) Shannon1948Shannon, CE. 1948. A mathematical theory of communication A mathematical theory of communication. Bell System Technical Journal27.
- Tishby . (1999) Tishby1999Tishby, N., Pereira, FC. Bialek, W. 1999. The Information Bottleneck Method The Information Bottleneck method. 37th Annual Allerton Conference on Communication, Control and Computing. 37th annual Allerton conference on communication, control and computing.
- White . (2017) White2017White, A., Malt, BC. Storms, G. 2017. Convergence in the Bilingual Lexicon: A Pre-registered Replication of Previous Studies Convergence in the bilingual lexicon: A pre-registered replication of previous studies. Frontiers in Psychology7.
- Xu . (2016) Xu2016Xu, Y., Regier, T. Malt, BC. 2016. Historical Semantic Chaining and Efficient Communication: The Case of Container Names Historical semantic chaining and efficient communication: The case of container names. Cognitive Science4082081–2094.
- Zaslavsky . (2018) Zaslavsky2018Zaslavsky, N., Kemp, C., Regier, T. Tishby, N. 2018. Efficient compression in color naming and its evolution Efficient compression in color naming and its evolution. PNAS115317937–7942.