“Someone spoke to the King in Zheng”: using high tech methods to solve an ancient Chinese mystery

A nice old fashioned detective story with intrigue, poison and machine learning

This story begins with an ending.

The Han Feizi is a compendium of essays about the bloodthirsty politics of the Warring States era in China. It opens with the death of its author in approximately 233 BC.

In the first chapter, we meet Han Fei[1] — writer, tongue-in-cheek contrarian, freelance politician and perennial insider-outsider — as he is about to speak before the court of King Zheng of Qin. Zheng’s armies are on the verge of sweeping across China, which he will rule under the name of Qin Shihuang, and Fei’s essays have already caught his attention. This is the perfect moment to secure a position in the Qin government. However, Zheng is too smart to fall for the sort of bland flattery that could be deployed elsewhere to win a sinecure. Instead, Fei stands to speak and delivers a ruthlessly effective criticism of Qin’s failure to wipe out the other feudal states, including Han — to whose ruling family he himself belongs. In a speech that is unquestionably one of the high points in the history of human rhetoric, he argues that Qin’s prior hesitations in overrunning its neighbours were a false mercy, merely prolonging a conflict that could have been brought to an end at multiple points over the past century if Qin’s advisors had only been more intelligent or less compromised in their allegiances. Having long been a critic of the Han government, Fei finally throws his family and his state to the wolves, aligning himself once and for all with Qin.

Maybe.

In the second chapter, however, things become far more ambiguous. King Zheng is planning to launch a final destructive attack on Han, but now Fei argues that this is the wrong approach. Han is too weak to present a threat and has fallen into a habit of subordination to Qin in any case. However, any attack on Han is likely to galvanise the anti-Qin alliance. A better strategy, he suggests, would be to bribe the southern state of Chu to remain neutral and attack Zhao in the north — a far larger and stronger state. While a successful attack on Han would strengthen the other states’ opposition to Qin, defeating Zhao would demoralise them such that their surrender could be assured “by official dispatch”. It is a convincing argument, and the chains of causality upon which it is based have an ineluctable algebraic quality. The conclusion seems natural and unarguable.

It is at this point that Li Si, Chancellor of Qin, stands up to speak. Having risen through the ranks via a mixture of academic ability, street smarts and killer instinct, Si seems to have known Fei of old. Indeed, Si appears to have been instrumental in securing Fei’s original invitation to Qin. One can only imagine, then, his reaction to Fei’s initial speech, which came dangerously close to accusing him personally of providing bad advice out of disloyalty — a capital crime. His reply is short and devastatingly brutal, with none of the carefully-calibrated persuasive technique that preceded it. Nevertheless, it lands blow after blow: “Fei came here most probably with the intention to elevate his own position in the Han Government by demonstrating his ability to save the Han State. By means of eloquent speeches and beautiful phrases he embellishes lies and falsifies plots in order thereby to fish for advantages from Qin and watch Your Majesty’s mind on behalf of Han. Indeed, if the friendship of Qin and Han becomes intimate, Fei will be esteemed; which is his self-seeking scheme.”[2]

To test the truth of this, Si suggests dispatching a diplomatic mission to Han. If Han is as compliant as has been suggested, they should be possible to convince the King of Han to travel to Qin in a display of submission, whereupon he can be detained and his state annexed. The plan is subsequently tried, with Si personally serving as the ambassador to Han. At this point he cannot lose: if the mission succeeds he will get the credit for his persuasive skills, while if it fails he will have rid himself of an irksome internal rival. Safe in this knowledge he pushes his advantage, addressing Han with explicit and undiplomatic aggression, doing his best to secure a panicked rejection of his offer.

Refused an audience with the King, he submits a succession of requests, alternating sarcastic parodies of Confucian courtesy with unveiled threats, humbly offering up his own life as a gage of his good faith, knowing all the while that the life that is actually on the line is Fei’s: “Now thy servant, Si, is sent here by the King of Qin, he is not granted an audience. Therefore, he is afraid the present chamberlains of Your Majesty have inherited the scheme of the former wicked ministers and might once more cause Han territorial losses. If thy servant, Si, is granted no audience while here and has to go home to report to His Majesty the King of Qin on his mission, the relations between Qin and Han will certainly be severed. Thy servant, Si, has petitioned for an audience only to present his stupid counsels inside the court and then to be chopped into inches to death outside the court. Thereon may Your Majesty deliberate!”[3]

The Han Feizi version of the story does not record the outcome; its readers already knew. As Sima Qian put it, in a paragraph that is a masterpiece of delicate ambiguity: “The King of Qin instructed officials to pass sentence on Han Fei. In the meantime, Li Si sent men to bring poisonous drugs to Han Fei and order him to commit suicide. Han Fei wanted to plead his own case before the Throne and vindicate his innocence but could not have an audience with the King. Later, the King of Qin repented and instructed men to pardon him, but Fei had already died.”[4]

This introduction leaves us with numerous nagging questions. Why is Han Fei seemingly serving as his country’s official representative in Qin, given that he had long been among his government’s loudest critics? Had he really decided to betray his family and ally himself with Qin, or are his speeches part of a long con aimed at mitigating Qin’s violence against Han? Why does Han refuse to receive Li Si’s embassy?

We would like to propose answers to some, if not all, of these questions.

We set ourselves the goal of solving the mystery, not by digging up long-lost books, but by subjecting well-known texts to modern investigative techniques. In particular, we focused on a particular book: the Stratagems of the Warring States. This text was compiled during the Han Dynasty by the archivist Liu Xiang and brings various disparate anecdotes together into a collection aimed at providing examples of successful strategic reasoning and persuasion. The original stories come from numerous authors, and while many of them are anonymous, it is often possible to track particular writers across the volumes via their individual stylistic quirks. Thus the reader becomes familiar with individuals’ authorial preferences and even comes to guess somewhat at their historical and political backgrounds, labeling them with made-up titles in the absence of names: “Wei Ran’s client”, “Chen Zhen’s client”, “the Grand Historian of Han”, “the Zhao satirist” etc. This is often more of an intuitive process than a rational one, and would be difficult to replicate using computer methods.

There is one exception, however. This exception concerns the author of a chapter known as Speaking to the King in Zheng, who — we have come to suspect— was not an anonymous bureaucrat, but rather Han Fei himself.

In this chapter, which is relatively short and can be read in its entirety here, an anonymous speaker in Han argues that successful political enterprises are often the product of fruitful partnerships rather than individual genius, with one party providing the force and the other handling public relations. He argues that Han could form just such a partnership with Qin, and suggests that it is somewhat incredible that no one has yet seized this opportunity: “In the past, Duke Huan of Qi brought the sovereign lords together nine times, but never once failed to respect the orders of King Xiang of Zhou. Thus it was King Xiang who had the respect and Duke Huan the power. If the sovereign lords respected Duke Huan enough to come nine times when he summoned them to meet, it was because of the respect he himself showed for King Xiang. Nowadays they say that the Son of Heaven could not find such a person and put him to work, even if he gave them the powers of Duke Huan. If this is so, it is because we ourselves failed to take on the role. How could this be down to anything but our strategic inadequacy or our failure to understand the rules of respect?”[5]

The first striking thing about this text are the stylistic similarities with the Han Feizi. Han Fei’s voice is one of the most distinctive in classical Chinese literature, and this chapter shares many of his hallmarks: the extensive use of first-person pronouns, the self-contained algorithmic quality of the arguments, the waspish sense of humour… Indeed, the similarity is so glaring that we expected to find dozens of commentaries discussing the parallels, and were surprised to discover none. Nevertheless, all of these things would be imitable by a sufficiently gifted writer. (The Stratagems itself contains a merciless parody of the genre in the form of the Su Qin Begins Advocating for a Horizontal Alliance chapter.) Thus, these indicators do not necessarily signify much alone.

What is much harder to imitate, however, is the striking lack of anything approaching a self-preservation instinct on the part of the author. One key quality that this chapter and the first chapter of the Han Feizi have in common is that fact that they are both beautiful, original and convincing demonstrations of the author’s point, but only to objective outside observers. In practice, each would have been almost guaranteed to drive its real-world audience into a murderous rage.

In the first case, Han Fei, either oblivious or indifferent to his surroundings, implies that Li Si — a man who had just secured him an enviable opportunity and who was presumably present in person during this interview— is either stupid or disloyal, meriting a grisly execution. In such circumstances, it is less surprising that Si wished to kill him than that he had the self-control to refrain from doing so there and then. In the second case, the speaker has the good grace to include himself among the list of those who have lacked the intelligence or the chutzpah to secure Han’s role as kingmaker, but his opinions on the Han cabinet’s collective ability would scarcely have endeared him to his colleagues. In each case the impression given is not so much that the arguments were intended as an expression of contempt for any particular listeners, but simply that Fei was transfixed by the abstract order of strategic calculus, and it did not occur to him that others may have different priorities. Either that or he was simply too used to exerting his personal charisma to get away with saying things that would see anyone else executed. The same reckless candour can be seen throughout the Han Feizi, and seems likely to have been the cause of its author’s stalled political career in Han. This attitude would have been far harder for a later writer to imitate than simple stylistic tics such as the use of 我 and 吾 as indefinite pronouns.

The circumstantial evidence is also strong. Speaking to the King in Zheng is immediately preceded by a speech (Speaking to the King of Han) that is also unattributed, but which is close in tone and style to the oratory of Su Qin, as the commentaries note. This chapter describes a desperate last-ditch attempt to hold together the anti-Qin alliance, of which Su Qin was the principal proponent. Moreover, while it is too short to lend itself to statistical analysis techniques, it incorporates several of Su Qin’s favourite rhetorical devices. It mentions multiple figures from the early historical records (Boyi, King Jie and King Zhou), it focuses on defensive rather than attacking strategies, and it emphasises the fact that if the listeners do not act now it will soon be too late to save their state. If this chapter is indeed an argument presented by Su Qin, then it would normally be followed by a counter-argument presented by Zhang Yi, his counterpart on the pro-Qin side. Why not, then, simply interpret Speaking to the King in Zheng as an unattributed Zhang Yi speech? The problem is that the speaker gives the distinct impression that he was an insider at the Han court. Firstly, he displays an anecdotal knowledge of details of the assassination of Chancellor Xia Lei of Han that do not appear in other versions of the story. Secondly, he speaks as though he were part of a single collegiate body with the King’s other advisors. While Zhang Yi had a gift for ingratiating himself among foreign elites (as he does in the Zhang Yi Runs Out of Money in Chu chapter, for example), there is no evidence for him having done so in Han.

Compounding this is the knowledge that most — if not all — of Zhang Yi’s actual rhetoric has been lost. Several of Zhang Yi’s speeches in the Stratagems are known to be forgeries or substitutions from other works, mentioning events that took place well after the death of the historical Zhang Yi. And — critically — at least one of them was originally written by Han Fei (see Zhang Yi Exercises his Persuasions on the King of Qin, footnote 1 for details of this).

Thus we hypothesise that Speaking to the King in Zheng probably began its life as a record of one of Han Fei’s speeches. A compiler — either Liu Xiang or a colleague — substituted it for a lost Zhang Yi speech in order to ensure that the section would fit the narrative format of the Stratagems, which includes an argument/counter-argument pair by Su Qin and Zhang Yi in every section. The original was then lost from the accepted Han Feizi corpus, and so the text survived only as a record of an anonymous politician’s speech to the court of Han in the Stratagems of the Warring States.

This is not merely idle speculation, however. To check this hypothesis we used a cosine similarity scoring algorithm to compare Speaking to the King in Zheng to 10,000-character excerpts taken from the Stratagems and the Han Feizi.[7]

The Speaking to the King in Zheng chapter was found to be closer to the Han Feizi text than to any section of the Stratagems, including the other Han chapters:

While classical Chinese was not punctuated, our initial instinct was to leave the modern punctuation marks in, on the basis that they go some way towards reflecting the length, structure and complexity of the phrases used, even if they were not part of the original text. Nevertheless, we also used the Ctext Tools to compare the results when punctuated and unpunctuated texts were analysed.

While the overall similarity scores change somewhat, the same trends are visible.

When the outputs are viewed in matrix form, the extracts from the Stratagems, despite being by many different authors, display a high degree of internal consistency, with similarity scores over 0.9 when analysed with punctuation and over 0.8 when analysed without.

It is not simply that two groups each have a high degree of internal similarity, however. When texts from one group are compared with those from the other, the average similarity score drops significantly. In other words, not only are the two groups internally consistent, but they are also clearly different from one another.

If the “Speaking to the King in Zheng” chapter is assimilated with the Han Feizi chapters, internal consistency drops a little (part of this effect is simply due to length, though it is not clear how much), but if we lump it in with the Stratagems chapters it drops by a lot more. In other words, “Speaking to the King in Zheng” seems closer to the Han Feizi texts when taken as a whole than to the Stratagems texts.

In an attempt to check that this was not merely a fluke, we also compared Speaking to the King in Zheng with 10,000-character sections from a variety of other classical Chinese texts: the Mozi, selected chapters from the Records of the Grand Historian (specifically, the Qin Shihuang, Li Si and Lü Buwei biographies), the Book of Lord Shang, the Xunzi, the Zhuangzi, and the Kongcongzi (which includes a story that incorporates several sentences from the Speaking to the King in Zheng chapter).

Once again, we see a greater degree of similarity between Speaking to the King in Zheng chapter and the Han Feizi segment than with any of the other texts.

It is interesting to note in passing that the text that came closest was the Xunzi, under whose author, Xun Kuang, Han Fei is supposed to have studied. (It is also interesting to note that a segment of Xun Kuang’s writing is featured in another Stratagems chapter — “A Visitor Persuades Lord Chunshen” — in which it is amalgamated with a section from the Han Feizi to create a single essay.)

However, we also wanted to get a clearer idea of the effect that rare and topic-specific characters were having on the final output. If two different authors write on the same topic, this will produce a relatively high similarity score. It doesn’t meant they’re the same person, they’re just using a lot of topic-specific vocabulary. Therefore we also attempted to separate out the effects of topic and style on the final output, and — ideally — to eliminate the former.

To control for the effect of topic on the similarity scores, we used a Manhattan distance algorithm. Firstly we clustered the full texts (including topic-specific words). As you can see, there are two clear clusters — a Stratagems cluster and a Han Feizi cluster, with “Speaking to the King in Zheng” being more similar to the latter.

We then removed all characters that did not appear in every text, a somewhat hacky way to minimise the effect of topic on the final results. Only the following characters remained:

Once the effect of topic-specific characters is eliminated, “Speaking to the King in Zheng” remains closer to the Han Feizi texts.

In other words, the author of “Speaking to the King in Zheng” uses similar basic phrasing to the author of the Han Feizi, even when the topics on which they are speaking differ.

While we may never be able to pin down the author with 100% certainty, we can say with confidence that all of the statistics we have currently indicate that “Speaking to the King in Zheng” is more similar to the early chapters of the Han Feizi than to the later chapters of the Stratagems, and that this effect remains present when the focus is on authorial style rather than topic.

And this is all very well, but even if this chapter has drifted from one corpus to another with the passing of time, why does it matter?

Because if it is indeed a chapter missing from the Han Feizi, then it must be describing events that happened prior to the current first two chapters. Thus it provides the historical background to the episode that they describe, explaining Han Fei’s reasoning in undertaking the mission to Qin and suggesting that rather than being on one side or another, he was on both.

In the context of the times, this is not as unusual as it may seem. As other works from the same era attest, it was not uncommon for leaders to accept advice from members of the enemy camp. These advisors knew that their associations were liable to count against them, and so rather than trying to finesse their target into going against his own interests, they attempted to find a course of action would benefit both parties to such an extent that it would be unreasonable to reject it. To take one famous example, Han originally sent Zheng Guo to Qin to propose the construction of a canal connecting the Jing and Luo rivers, on the basis that this would strengthen Qin’s economy but also divert resources that would otherwise have been used to attack Han. The Qin authorities found out that this was Han’s goal, but continued to employ Zheng Guo on the project because the economic benefits were irrefutable, whatever the original motivations behind the proposal.

Han Fei seems to have decided that the best way for Han to survive was by becoming a partner to Qin, rather than an antagonist. Han’s alliances vacillated frequently, and Fei’s own influence within the court hierarchy was weak at the best of times, so his ability to engineer such an outcome would have been circumscribed. Nevertheless, he appears to have decided to roll the dice, betting on Han’s previously well-demonstrated willingness to fall in line behind the strongest foreign power, whatever his own status in the affair. However, he failed to anticipate the degree of subjection that would be demanded by Si. Goaded by previous slights and the continuing threat to his career, Si deliberately proposed to make excessive demands of Han, knowing that this approach would leave Fei in an impossible position. If Han accepted his demands, Si would gain all the credit, having led the embassy himself. If it rejected them, Fei’s advice would be shown up as faulty and his life would be in danger.

Fei’s position was a bad one, but with some finesse he could undoubtedly have talked his way out of it, ensuring his own survival — if not his state’s — as long as no other misfortune intervened. Such luck eluded him. The Han leadership, expecting a friendly approach and struck dumb by Si’s calculated threats, not only failed to comply with his demands, but were panicked into the kind of undiplomatic silence that could easily be read as a preparation for war. Clearly their mission to Qin had gone disastrously wrong, though they could not have been sure exactly how or why, and by this point they would have been far more concerned about saving themselves than with rescuing an errant diplomat who had never ranked particularly highly in their affections in any case. Realising that his plan was working, Si fired off increasingly belligerent messages, provoking only more bewildered silence by way of reply. Thus Qin concluded that Fei was guilty of failing to deliver on his promises and should suffer the consequences. Per his own precepts: “Whenever a minister utters a word, the ruler should in accordance with his word assign him a task to accomplish, and in accordance with the task call the work to account. If the work corresponds with the task, and the task corresponds with the word, he should be rewarded. On the contrary, if the work is not equivalent to the task, and the task not equivalent to the word, he should be punished.”[9]

Joseph Lim, Vincent Arlen Santoso and Jennifer Dodgson contributed to this article. To contact the authors, please email jen@lexikat.com, or visit lexikat.biz.

[1] To aid clarity, we will be referring to the book as the Han Feizi and the author simply as Han Fei. Other works adopt different conventions. Elsewhere, we adopt the Warring States convention of referring to people by the first names rather than their last.
[2] W. K. Liao’s translation, with the names changed to follow the conventions of Hanyu Pinyin.
[3] W. K. Liao’s translation, with the names changed to follow the conventions of Hanyu Pinyin.
[4] W. K. Liao’s translation, with the names changed to follow the conventions of Hanyu Pinyin.
[5] Author’s own translation.
[6] There is no online translation of this passage. The original can be found here.
[7] We cleaned the test data by removing all personal and place names from the Speaking to the King in Zheng chapter, and cut the Han Feizi text down to include only the first four books (minus the two Li Si speeches mentioned above) with the aim of including only essays and speeches that seem reasonably likely to have been written by the historical Han Fei. Because this is still much longer than most sections of the Stratagems, we extracted two non-overlapping 10,000-character sections. We then took sections of a similar length from the latter part of each book of the Stratagems that was long enough to permit this, with the latter parts chosen specifically to target the texts that reference events closest in time to the posited date of the “Speaking to the King in Zheng” chapter. The Speaking to the King in Zheng chapter and the Zhang Yi Persuades the King of Qin chapter were removed to prevent authorial overlap, as was the A Vistor Persuades Lord Chunshen (客說春申君) chapter from the Chu section, which contains blocks of text from the Xunzi and the Han Feizi. All data was segmented on a character-by-character basis. The results presented here were the result of running the code with no stop-word list applied. In this comparison we look only at term frequencies (TF), preferring not to eliminate or minimise the impact of frequently-occurring characters by applying IDF weighting. We made this choice partly because this approach tends to perform less well on short texts such as these, and partly because some of the most frequently occurring characters are also among the most useful for working out the origin of a text. Thus, even though 楚 (referring to the state of Chu) is a frequently-used character, it happened to be subject to a naming taboo in Qin relating to its use as part of King Zhuangxiang’s given name, and was replaced by 荊 (Jing, an alternative name for the state), and so can be used to help pin down a text as having been written for consumption in Qin during the second half of the third century BC. To deal with issues surrounding text length, all texts for comparison were cut down to the same length.
[8] W. K. Liao’s translation, with the names changed to follow the conventions of Hanyu Pinyin.
[9] W. K. Liao’s translation.