Indian scholar at Cambridge solves 2,500-yr-old Sanskrit algorithm problem in Panini’s text

Ashtadhyayi, which comprises rules to derive or form new words from root words, often has conflicting rules. Scholar finds way to resolve rule conflicts in the text as Panini intended.

PhD scholar Rishi Rajpopat | Pic courtesy: Cambridge University Library

Bengaluru: Rishi Atul Rajpopat, a PhD scholar at the faculty of Asian and Middle Eastern Studies at St. John’s College, Cambridge, has solved a grammatical problem posed by the texts written by ancient Sanskrit scholar Paṇini.   

Paṇini’s text Aṣṭādhyāyī, which comprises a set of rules to derive or form new words from root words, often contains conflicting rules for creating new words, with many scholars confused about which rules to use. 

Solving such conflicts in this linguistic algorithm of a book was the subject of many scholars’ interests. Paṇini himself wrote a meta-rule to resolve rule conflicts, which so far scholars have interpreted as: In the event of a conflict between two rules of equal strength, the rule that comes later in the grammar’s serial order wins.

In his dissertation, Rajpopat argues that this metarule was historically misunderstood — instead, what Paṇini meant was that between rules applicable to the left and right sides of a word, he wanted the reader to choose the rule applicable to the right side. 

With this logic, Rajpopat finds that Paṇini’s algorithms produce grammatically correct words and sentences without errors. 

For example, in the sentence jñānaṁ dīyate guruṇā — knowledge (jñānaṁ) is given (dīyate) by the guru (guruṇā) — there is a rule conflict when trying to form the word guruṇā, which means ‘by the guru’ and is a known word. 

The word comprises the roots, guru + ā. Going by Paṇini’s rules to create the word that will mean “by the guru”, two rules become applicable — one to the word guru, and one to ā. This is resolved by picking the rule that is applicable to the word on the right, resulting in the correct new form guruṇā. 

Rajpopat’s work is a rebuttal to scholars over the course of two and a half millennia. 

Scholars who tried to resolve Paṇini’s rule conflicts include Jayaditya and Vamana in their commentary treatise Kāśikāvṛttī, Patanjali in his Mahābhāṣya, and Katyayana in his Vārttikakāra.

“After Pāṇini wrote his text, Kātyāyana expounded upon it, followed by Patanjali,” Rajpopat told ThePrint. “Several months after I started working on my thesis, I discovered that Kātyāyana had also deduced the same logic in a remote corner of his work. However, he too seemed to have decided to use alternate interpretations for rule conflict as well. Since in the Sanskrit tradition, scholars build up on the previous expert’s work more than the canonical text, this interpretation of the rule seems to have fallen through the cracks.”

Panini’s algorithmic texts

Paṇini, the definitive linguist of his time, wrote the authoritative and comprehensive text on Sanskrit grammar. 

In the Aṣṭādhyāyī — with eight segments as the name suggests — he outlines rules to produce variations of a root word in such a way that they are grammatically and syntactically correct according to the rules of Sanskrit. These include grammatical rules for the formation of new words, such as sandhi or joining two words to produce a third. 

In all, about 4,000 rules are compiled in the text as sutras, and each book or segment outlines a step-by-step method — like an algorithm checking for conditions — to produce and construct new words. But the sutras are not always obvious in their meaning. Since they are concise and made up of limited words, they can be confusing to modern readers. 

Additionally, these sutras are not stand-alone — each is built upon the preceding set of rules in a linguistic device called anuvr̥tti, or the continuation of one rule into the next. This means that sutras use recall and reference keywords that were used previously. 

A page from Panini’s writings | Courtesy: Cambridge University Library

Sometimes, it is also confusing whether a sutra references any previous rules, and sometimes, it is unclear whether a rule needs to be continued to the next. More often than not, even when a rule is understood in its meaning, it is unclear where to apply it. 

Perhaps most confusingly, when a linguist knows where to apply one rule, another conflicting one also becomes applicable at the same step. This leads to one rule blocking the other, or both blocking each other. This rule conflict is where Rajpopat’s thesis comes into the picture. 

As an example, Rajpopat mentions the words formed by the root vr̥kṣa. Combining the words vr̥kṣa and bhyām produces vr̥kṣābhyām, where the last ‘a’ sound of the first word is replaced by a longer ‘a’ sound. On the other hand, combining vr̥kṣa and su gives vr̥kṣeṣu where the same sound is replaced with ‘e’. Now, when the word vr̥kṣa has to be added to the plural bhyas, should the ‘a’ be replaced with a longer ‘a’ or ‘e’? 

Various scholars had argued that different rules were to be interpreted in different ways, but Rajpopat’s Occam’s Razor solution gives the correct form, vr̥kṣebhyaḥ.

Rajpopat’s work solves simpler conflicts, as illustrated by the guru example above, and more complex ones.

To illustrate the more complex rule conflict that Rajpopat solved with an example, consider the word for “from god” which are split into their roots deva + bhis.

When these two words are to be joined together, there are two rules that conflict:

Rule number 7.3.103 applies to the word ‘deva‘ and states that when the words are joined together, the ‘a’ should become ‘e’, to form devebhih.

Rule number 7.1.9 applies to the word bhis and states that the ‘a’ sound preceding bhis should be replaced with ‘ais‘, making the new word devaiḥ.

Under such a circumstance, previous scholars have interpreted the metarule to mean the later sequential rule, ie., 7.3.103. But devebhih is not the correct word, devaiḥ is. Scholars previously then interpreted this conflict as being an exception — in cases where the applicable rule provides a wrong result, the other rule needs to be applied.

Panini’s meta rule states vipratiṣedhe paraṁ kāryam, which traditional scholars have interpreted as ‘in the event of a conflict between two rules of equal strength, the rule that comes later in the serial order of the Aṣṭādhyāyī, wins.’

Rajpopat went back to the canon text and reinterpreted the meaning of the word “para” to ‘right hand side’. By his interpretation, the rule 7.1.9 applies, making the new and correct word, devaiḥ.

This solution does away with a number of creative and complex linguistic workarounds that others have proposed in the past to resolve such rule conflicts.

 In a statement, Rajpopat’s supervisor Vincenzo Vergiani said, “My student Rishi has cracked it — he has found an extraordinarily elegant solution to a problem which has perplexed scholars for centuries. This discovery will revolutionise the study of Sanskrit at a time when interest in the language is on the rise.” 

Rajpopat recalled how he managed to arrive at his conclusion. “I had a eureka moment in Cambridge. After 9 months trying to crack this problem, I was almost ready to quit, I was getting nowhere. So I closed the books for a month and just enjoyed the summer, swimming, cycling, cooking, praying and meditating. Then, begrudgingly, I went back to work and with,in minutes, as I turned the pages, these patterns starting emerging, and it all started to make sense. There was a lot more work to do but I’d found the biggest part of the puzzle,” Rajpopat said in the statement. 

He worked for another two and a half years on the project before finishing his thesis.

Sanskrit for computers

While Sanskrit is often regarded as a “language suitable for computers”, it’s just a language like any other.

However, Paṇini’s comprehensive texts could prove useful for natural language processing (NLP), where computers contextually understand natural language and its nuances. Machine learning algorithms struggle with aspects of NLP such as natural language understanding and generation. A rule-based approach, such as that of Aṣṭādhyāyī, can find potential use in training AI systems for NLP. 

Paṇini’s work is not the first one to delve into detailed grammatical rules, but it builds on previous work. His treatise is one of the oldest texts written in and about the Sanskrit language that has survived entirely, and is also linguistically one of the oldest in history. It was written in the Classical Sanskrit form, and in fact, marks the beginning of the Classical Sanskrit era. 

Vedic Sanskrit preceded Classical Sanskrit, and texts written in Vedic Sanskrit were accompanied by Prātiśākhyas, which explained how to pronounce the texts in the Vedas. These supplementary texts were then expanded upon by scholars, eventually leading to writing comprehensive texts about grammar, Pāṇini writing his masterpiece, and then interpretation of Pāṇini’s work, as done by Katyana and Patañjali (one of several ancient scholars of the same name), in the 2nd century BCE. 

Sanskrit, which is a combination of the words sáṃ (‘together, good, well, perfected’) and kṛta– (‘made, formed, work’), was comprehensively edited by scholars and refined as a language when naturally evolving languages or Prakritic ones were being still spoken. As a result, the language consists of well-laid out grammatical rules, which were first compiled by Paṇini. 

Even today, Aṣṭādhyāyī stands out as a comprehensive and complete work that algorithmically describes a language and how to form new words in the language without misinterpretation. This lends itself to be machine-like in nature and, thus, is useful for training NLP models. 

“Computer scientists working on natural language processing gave up on rule-based approaches over 50 years ago,” said Rajpopat. “So teaching computers how to combine the speaker’s intention with Paṇini’s rule-based grammar to produce human speech will be a major milestone in the history of human interaction with machines, as well as in India’s intellectual history.”  

(Edited by Tony Rai)