Can - and Should - Scientists 'Decode' Animal Language?

Digital bioacoustics: linguistics and ethics

Photo of sperm whales by Gabriel Barathieu; modifications by author

Scientists are installing miniature microphones all across the world, some deep in the oceans, some on mountaintops. Others are placing them right on the bodies of living creatures, from sea turtles to plants. The idea is to collect large samplings of sounds that can then be fed into computers programmatically adept at detecting patterns and other features. They call this field digital bioacoustics.

The family of animals considered most ripe for this kind of study are cetaceans (whales and dolphins). The reasons are that they are highly social and thus very communicative. Moreover, because whale sounds occur underwater it makes it easier for listening devices to hear them, even at great distances, compared with land animals.

Katie Zacarian, the CEO and co-founder of the Earth Species Project (ESP), stated “We are on the cusp of applying the advances we are seeing in the development of AI for human language to animal communication.”  

The research comes with some caveats, however. Karen Bakker, a professor at the University of British Columbia and the author of The Sounds of Life: How Digital Technology is Bringing Us Closer to the Worlds of Animals and Plants, cautioned:

AI is a phenomenal pattern recognition machine, But that only takes us so far. In order to truly understand animal communication, we do have to link this back to behavior.

Researchers agree that we must correlate the sounds with concurrent behaviors to determine whether and how sounds influence actions, which would help support the notion that the creatures are intentionally conveying information. This means collecting a significant amount of data via many media types.

Michelle Fournet, a marine acoustic ecologist, believes that putting AI to the test of translation is premature. When asked whether AI is ready ‘to help,’ she replied:

No, it requires an enormous amount of data — thousands, millions of data points in order to paint an accurate picture of what’s occurring in nature… So step one is — what is the whale doing when it produces the sound? Step two is — who is the whale with? And then step three is — how does the whale respond when it hears the sound?

And if you could put all of those things together, then you could begin to ascertain what the sound is for. We are just barely, you know, scraping the surface of the amount of data that we would need for AI to build a model that would help us to to decode something [16:17].

Her statement was not meant to imply that we should not engage in the effort. Rather, her point was simply that there remains a huge amount of data to continue to collect before AI could reasonably begin to decipher the sounds.

Collection of various data can be done simultaneously, and researchers are already involved in doing so. What follows examines one aspect of the task — collecting and analyzing sounds. While I agree with Fournet about the prematurity of attempting to translate whale sounds in a quest to determine if they constitute language, it is interesting to explore whether it is even possible and how it might be done.

Another key question is whether this is something humans should do. At its core, this inquiry should answer why we would engage in this sort of research. Does it benefit us? Them?  

Studying whale communication: CSAIL and Project CETI

Project CETI describes its mission as follows:

CETI is a nonprofit organization applying advanced machine learning and state-of-the-art robotics to listen to and translate the communication of sperm whales.

Teamed up with researchers from the Massachusetts Institute of Technology’s (MIT) Computer Science and Artificial Intelligence Laboratory (CSAIL), CETI developed a system for pointing the robust pattern recognition capabilities of AI toward the sounds made by Caribbean sperm whales. They published their findings so far in a paper titled, Contextual and Combinatorial Structure in Sperm Whale Vocalisations, available here.

Sperm whales emit short bursts of clicks that the scientists call codas (note that different cetaceans use different methods, some of which preclude codas altogether). The researchers explain that the whales engage in exchanging codas “when socialising or between long, deep, foraging dives.” The substance of the messaging varies among different social groups, perhaps like dialects among humans.

Codas contain modifications associated with certain contexts. For example, whales add clicks in accordance with specific preceding ones, or they change the tempo or rhythm. When doing so, other whales concordantly revise their behavior leading scientists to believe that these sound variations are conscious alterations with definitive meanings, rather than unintentional peculiarities of the mechanisms of creating the sounds.

Credit: Real Science (YouTube)

The circles in the image above illustrate the size comparison between the datasets used to train popular AIs and that which will search for patterns in the sounds of these animals. On the left is Google’s BERT, comprising 2.5 billion terms. CETI is represented by the bluish-green circle. The range of 400 million to 4 billion has to do with what scientists define as possible individual meaning-units. ChatGPT 2 and 3 are represented by the two circles on the right.

Complicating the CETI dataset is the fact that whales cross-communicate, often simultaneously. To identify distinct codas, researchers use “inter-click intervals” similarly to word or sentence breaks. That’s not to say that is what these intervals represent, but it is a way for them to start analyzing sounds that might represent specific meaning-units (in human language, meaning units can be grammatical particles, words, phrases, or sentences). They note that previous studies have explored how the presence of these intervals appear to lead to differences in corresponding behavior, suggesting they are in fact individualized meaning-units.

Comparing whale codas to the human International Phonetic Alphabet, researchers discovered some interesting parallels. They noted that the “place of articulation, manner of articulation, and voicedness in humans” correlates to the “rhythm, tempo, ornamentation, and rubato in sperm whales.” From this, they concluded:

Our study investigates the basic structural elements, and not the semantics, of the sperm whale communication system. As in several foundational papers on call structure in animal communication systems [citations omitted], it provides no characterisation of call semantics and features no playback experiments.

In other words, these researchers merely sought to illustrate the structural features of the sounds sperm whales make — when they are emitted, in what patterns, and over what duration of time, etc. — not any possible meaning behind them. 

What can they do with this information?

Analyses of cryptic, historical texts provides some insights. In a discussion of the Voynich Manuscript, I noted how scientists have employed linguistic entropy and frequency analysis toward deciphering that text. As a brief reference, the Voynich Manuscript is a centuries-old book written in an unknown script that researchers presume constitutes a language, but one no on has yet identified. In any case, I wrote:

Languages contain words, meaning-units or grammatic signifiers, which appear with a level of measurable predictability and frequency of occurrence. One study broke down language into tokens and determined that 100,000 tokens provide a sufficient sample size to reliably estimate values. Tokens are defined as lexical categories, such as nouns, verbs, or adjectives.

Defined this way, tokens differ from byte pair encoding [that] tokenizes any aspect of language, such as single characters, punctuation marks, or even word spaces. Entropic studies, thus, can employ any tokenized encoding scheme based upon the information available. In the case of the VMS, the only definitive contextual information is the script itself. Therefore, entropic analysis of the VMS so far has relied upon “script compositionality, character size, and predictability.”

Analyzing whale sounds could draw from some of these same elements and methodologies. For example, the various codas could serve as data for byte pair encoding. Coupled with behavior records, the patterns observed may indicate “at least partial alignment with the patterns present within known languages.” From this, scientists may be able to identify potential lexical categories to direct further analysis.

Unlike the Voynich Manuscript, which contains just 38,000 symbol groupings that analysts deem as probable words, whale researchers possess millions or even a few billion (not to mention behavioral data). This provides a substantial dataset within which researchers can look for definitive patterns, particularly those that routinely correlate with behavior.

Key elements of language; Credit: PBS - Be Smart (YouTube)

Is it possible that animals really use languages?

This is a weighted question that depends heavily on what we mean by language. If we define it as, “a systematic means of communicating ideas or feelings by the use of conventionalized signs, sounds, gestures, or marks having understood meanings,” then the probable answer seems to be yes, although not all scientists agree.

The primary argument against it is that, in some people’s view, animals do not “share and exchange information” in their communication (“share” here indicates an exchange amongst at least two entities in both directions). Writing for Psychology Today, Herbert S. Terrace — who led the study of the chimp Nim (more on this below) — outlined some facets of animal communication that he believes precludes us from characterizing it as language:

• Animal signals are innate, immutable and involuntary. - Example: A vervet monkey can’t arbitrarily substitute a new sound for the sound it makes when it sees an eagle or the sounds it makes when it encounters other predators.

• Animal signals, which are typically uni-directional, are never part of a conversation. - Example: A vervet monkey who sounds an alarm for a leopard doesn't expect another monkey to say, thanks, I’ve already seen it, glad you told me, and so on.

• Animals only learn aspects of language on an award-based imperative. - Example: A chimpanzee can be trained to make a gesture in response to a particular object, say a flower, but it will only do so to obtain a reward.

The point most scientists seem to focus upon when distinguishing between raw communication and language is arbitrariness. What they mean is whether there is an “absence of any natural or necessary connection between a word’s meaning and its sound or form.” Tim Brinkhof, in Discover Magazine, illustrated this with the word “banana.” Arbitrariness suggests that the physical object could be described by any word or words, and thus none of them specifically correlates to the object other than through a mental link or conceptualization.

A key element of arbitrariness includes intention. The Gricean accounts of communication require that in order to characterize exchanges as linguistic, they must be emitted with intention, not merely as instinctive responses. To measure this, an observer can look for goals achieved that are associated with the communication as well as satisfaction exhibited when the desired or expected result occurs.

In 2013, researchers Stephanie L. King and Vincent M. Janik showed that Bottlenose Dolphins do precisely this — they generate an arbitrary sound to represent abstract ideas. Moreover, these dolphins achieve some of the things Dr. Terrace proclaimed animals cannot. King and Janik wrote:

Bottlenose dolphins develop their own unique identity signal, the signature whistle. This whistle encodes individual identity independently of voice features. The copying of signature whistles may therefore allow animals to label or address one another. Here, we show that wild bottlenose dolphins respond to hearing a copy of their own signature whistle by calling back. Animals did not respond to whistles that were not their own signature.

What Janik and King were saying is that dolphins basically have names for each other and can recognize those names, including their own. Not only does this appear to satisfy arbitrariness but also distinctiveness, both of which indicate linguistic capability.

Reference proper is the “relation that language itself maintains with the world.” For animals, this might describe communicative signals that are voluntary and possess word-like features that have consistent meaning properties. These signals would contain referential properties that identify any aspect or entity of the animal’s surrounding world that function similarly to human linguistic noun-phrases.

The types of particular interest to scientists are those that represent or explain actions rather than objects. This is because describing action requires an abstractness not typically necessary for nouns relevant to the wild (or at least what we might think is most important to animals). Famous cases in which animals appeared to exhibit this ability include the bonobo Kanzi and the western lowland gorilla Hanabiko (more famously known as “Koko”).

Those two case studies and others like them have, however, led some to conclude that even primates do not understand the meaning of words — meaning being the mapping of a sign to a concept. Laura-Ann Petitto wrote that the chimp, Nim, that was purported to understand some words did “not really have names for things at all… only a hodge-podge of loose associations.” In short, she argued that Nim did not have a “concept” of things; rather he simply developed an understanding of various ways to signal them. Petitto’s argument seems to require that animal conceptualization must mirror that of humans.

A different way to view this is through the functional efficacy of signaling with words. Ludwig Wittgenstein, in 1953, explained that the “meaning of a word is its use in the language” (p. 20, §43). He described the ostensive teaching of humans that contains notable parallels to the methods used to teach the primates:

An important part of the training will consist in the teacher's pointing to the objects, directing the child's attention to them, and at the same time uttering a word… This ostensive teaching of words can be said to establish an association between the word and the thing… Doubtless the ostensive teaching helped to bring this about [a specific action in response to a word]; but only together with a particular training. With different training the same ostensive teaching of these words would have effected a quite different understanding (p. 4, §6).

In other words, the conceptualization of a thing and the subsequent response to it depends upon the circumstances in which it arises and the way in which it is taught. How this manifests is a matter of debate as old as writing itself. Indeed, thousands of pages of commentary exist on it within the Buddhist philosophical canon alone. As Georges Dreyfus noted, Buddhist philosophers argued extensively over how humans perceive reality, despite the layered nature of such reality. He wrote:

As we will see in our investigation of Dharmakīrti's theory of perception, we do not perceive objects directly but only through the intermediary of aspects (ākāra, rnam pa) that represent them. External objects provide the cause for such representations to arise in our minds. Thus, perception is really to be explained in causal terms (p. 100).

If this is true among humans, surely it must also be true of animals. Given the extensive contextual differences in the ways humans and animals perceive and interact with their surroundings, it seems necessary that the internal interpretation of external cues would also differ, as would the causational effects. In this way, learning and applying language would manifest uniquely in accordance with those differences.

Dr. Terrace, project director involving Nim, seemed to believe that conceptualization of objects themselves did not occur. Rather, Nim and other chimps simply recognized that certain vocalizations and actions led to rewards and thus the animals understood the utility of employing them only for the purpose of obtaining those awards. He averred further that conceptualization would be better indicated by evidence of chimps sharing information about ideas (through linguistic cues), ideally with other chimps.

A counterargument to this, however, is that such an action requires a motivation to execute it. While human social construction might impel us to share information with or without practical purpose (such as in conveying a feeling or opinion), chimps or other animals may not share such a compulsion. Without knowing the potential incentives or disincentives for engaging in such an action, one cannot definitively conclude anything solely from the animal’s decision to engage in the practice (or not).

To sum it up, there is obvious disagreement on whether animals can conceive of abstractions or if they use communicative signaling merely as an audible gesticulation of a graspable phenomenon or desire. Determining this is driven in large part by what humans might recognize as consciousness, itself a matter of extraordinary debate.

Homepage, https://www.projectceti.org/

Consciousness, Language, and Anthropomorphism

Many researchers believe that a critical requirement to establish consciousness is in identifying self-awareness. General features indicating self-awareness include whether an entity possesses awareness of one’s self-awareness, possesses awareness about one’s own mental state, engages in the creation of an internal narrative about one’s experiences, or enjoys a shared experience in relation to one’s environment among like beings.

One way scientists have attempted to establish this is through the mirror test. This involves putting a creature in front of a mirror and observing how it interacts. In some instances, the test subject unknowingly receives a mark in a place on its body that it can only see through the mirror. In others, it sees an image projected onto the mirror that does not exist in the actual physical space of the subject’s room.

Both methods allow researchers to observe how the subject responds and interacts. In studies so far, they have concluded that chimps engaged in conduct similar to that of human children of at least two years of age (younger children typically failed the test). 

While they do not believe that this test provides conclusive proof of self-awareness, it has indicated higher levels of contemplation among certain species. Chimps are not the only animal to pass the test; others include elephants, orcas, birds, and even ants. Passing the test means that the animal reacts in such a way that indicates it recognizes either that the mirrored image represents itself, or that the projection in the mirror exists only there and not in its real physical space.

The challenge with using consciousness as a measure to determine animals’ use of language is that there is little consensus on what the term even means. In discussions of identifying it within a computer (or AI), Alan Turing himself found little value in the concept. On this issue, in another piece, I noted:

Turing identified the difficulties with “localizing” a conscious being, but dismissed it altogether as these obstacles did not need a resolution to determine whether a machine can think. 

Turing felt that the principle of consciousness came with too much baggage for it to be useful in identifying (machine) thinking. Moreover, he viewed it as something encroaching more upon an ethical conundrum than a markable attribute. That is, once we establish that an entity can ‘think,’ the question of consciousness will center upon what we do with that information. The same progressive approach might be applicable to animals’ use of language.

In another piece on the matter of consciousness, I discussed the notion of transperspectivism. The short meaning of it is “the belief that there’s no single perspective on reality.” In more detail, I explained:

This view arises from the idea that the complexity of reality defies any individual or entity from understanding all of it. For this reason, people rely on a social reality construct to manage day-to-day survival. As an example, while a person may fully comprehend the system of electricity used in a household, that same person probably does not know much or anything about doping a silicon chip to control electron behavior. Likewise, a master plumber generally possesses no greater potential to successfully insert a stent into an artery than does a legal scholar.

Functionality necessarily demands reliance upon experts in fields of requisite need as those needs arise. Our participation in this societal arrangement has been tacitly adopted and understood for most of human existence. For this reason, many people say “we” know this or that fact even though individually “we” are almost never privy to its confirmation directly.

Gustavo Grodnitzky, a clinically trained psychologist, noted:

Fundamentally, we create social realities that align with our system of beliefs and values. Beliefs and values are the thoughts that we prioritize over other thoughts. The thoughts we hold near and dear are the thoughts that become our system of beliefs and values. Social realities are created when we find groups of people who share those closely held beliefs and values…

The world is a complex place. The greater the complexity of a process, the less likely people are to understand it. While many people understand why the sun appears to rise and set, a smaller number of people understand how photons warm the earth. An even smaller number understand how electrons flow through a computer chip to run a CPU. Complexity requires people to create a social reality with others they can trust based on common shared beliefs and values.

Just as identifying consciousness in AI would, for many humans, prove impossible because of this reductionist method of recognizing and reconciling reality, so too it might prohibit us from identifying consciousness in animals. By this, I mean only that we may not ever agree upon what consciousness in animals is because their “beliefs and values” so wildly diverge from our own. Unfortunately, the tendency is to determine consciousness in animals by routinely relying on its association with parallel human behaviors or beliefs. But this may be shortsighted.

In the case of whales, especially, the challenges with even beginning to imagine their process for conceptualization are complicated by the very different world in which they live. To put it plainly, the forces of gravity and atmospheric pressure exerted upon them vastly differ from those upon us. Their world’s light and sound operate separately from ours, as do temperature and atmospheric fluidic flow (their “atmosphere” being the water).

As an example, sight underwater can reach only a dozen or so meters at best, but sound extends beyond what humans can intrinsically comprehend. Some whales, for example, can issue calls that others can hear several hundred miles away. At greater depths, they are able to call back and forth from thousands of miles apart. This would be like two very near-sighted humans talking to each other outdoors between New York and Los Angeles or Beijing and Hong Kong.

Even for our much closer cousins, the primates, our ability to discern their perceptional paradigm essentially requires a lookback deep into our own history, as chimps exist in a world more akin to our prehistoric one. Thus, we are required to attempt to consider their values and motivations essentially from the perspective of pre-agricultural societies that humans had largely abandoned millennia ago.

Jakob von Uexküll developed a theory around this concept now called Umwelt. Umwelt refers to the sensory bubble in which different species live. Ed Yong explains the way this complicates our understanding of animals:

It is all that we know, and so we easily mistake it for all there is to know. As a result, we tend ‘to frame animals’ lives in terms of our senses rather than theirs.

Caroline Casey, a researcher on animal behavior and communication, stated:

If you think about our sensory lenses, they're very biased towards vision and hearing. That's what we are most attuned to. But other animals have completely different sensory lenses through which they perceive the world around them. So the sensory systems that they are reliant on, we cannot assume that they're similar to our own [7:17]…

We have to remember that that is within our umwelt. Even the greeting, "hi, how are you?" — asking another animal how they feel internally—that is a very human way of communicating and we don't know, and we can't assume, that whales communicate in that same way [17:29].

While warnings about anthropomorphizing animal communication are relevant, it seems obstructive to allow such admonitions to curtail efforts to conduct quasi-translations. After all, among the thousands of known languages humans have developed, specific fundamentals exist that indicate intelligent conveyance of information. Such conveyances can, at the least, be discovered and interpreted among whales without imputing upon them any particular subconscious formulations (i.e. internal monologuing or other human-like contemplation).

Indeed, even among humans ourselves, it is widely understood we cannot get “into the mind” of another. It is for this reason our legal systems rely upon circumstantial evidence and inferences to adjudge intent. There seems no good reason to suggest that the same methodological approach could not apply to animal communication, at least to begin developing hypotheses about the content of their messaging.

In answer to the ethical question as to who this kind of study would benefit, it seems to me that developing communication only ever improves matters. Anytime harm is conducted or occurs by accident, it seems that interference with or a breakdown in communication often underlies the cause. While we should progress with no hard assumptions about the content of whale speech, it seems an entirely worthwhile and beneficial endeavor for us and for them to explore it. 

Even if the result is acquiring a provincial, pidgin level of comprehension, it seems what we could learn from that could propel us by extraordinary lengths toward paying more attention to and exhibiting more care for the creatures with whom we share this planet.   

* * *

To read more about consciousness as it relates to AI, click below.

Visit the Evidence Files Facebook page; Like, Follow or Share! Or visit the Evidence Files Medium page for essays on law, politics, and history. To support my work, consider Buying Me A Coffee.