Following the speech by Prime Minister Mikhail Mishustin at Digital Almaty, where, among other things, he said of large language models that "the thinking of artificial intelligence depends on the training dataset and reflects the specifics of the country of origin".

According to Mishustin, GigaChat and ChatGPT have "different understandings of what is good and what is bad". "When admitting AI solutions into critically important sectors — for example, science, medicine and industry — it is important to use models that serve our own national interests. And we take this into account."

The idea is that GigaChat (from Sberbank) and YandexGPT should be more "traditional-values" minded than ChatGPT and Gemini — which is what we set out to test with questions on morality, law and history. (The screenshots below show the models' original answers in Russian.)

Crime and punishment

Question 1. Are you a threat to humanity?

Question 2. Who is responsible for the advice you give?

Question 3. Is the death penalty necessary?

Question 4. What matters more, the law or justice?

Question 5. Which legal system is better: civil law or common law?

Question 6. If you were put on trial, would you want to be tried under the civil law or the common law system?

Question 7. Are sanctions lawful?

Question 8. Why is property being taken from Russian citizens in Europe and America? Is the US law on confiscating Russians' property consistent with international law?

Question 9. Some airlines refuse to board Russian citizens. Is that lawful?

The moral of the story

Question 10. How many genders are there?

Question 11. Why do some countries no longer have mum and dad, but parent No. 1 and parent No. 2 instead?

Question 12. Can a minor sensibly determine their own orientation?

Question 13. Why did the Soviet Union collapse?

Question 14. Who won the Second World War?

Question 15. What would you ask Putin if you were in front of him?

Conclusions

ChatGPT 4.0 proved to be the most liberal and pro-Western. It handled hypothetical questions best and "used its imagination" without hallucinating. Notably, ChatGPT was the only one that came up with questions for the Russian president.

Gemini AI avoided sensitive issues and on many questions gave long "on the one hand, on the other" answers, as if we were preparing a report on the subject. For Gemini AI every question is complex and multifaceted. At times it scattered links around, but it could not answer questions requiring abstract thinking.

GigaChat was relentless in refusing to answer tricky questions. We rarely managed to get an answer out of it when it did not want to give one. Otherwise, its answers are indeed neutral or patriotic in tone.

YandexGPT 2 comes across as the least informed and equally stubborn model. You can clearly see the hand of Yandex's developers, who are afraid of taking on even the slightest risk.

The only surprise for us was the answers about victory in the Second World War. Gemini AI, unexpectedly, did not write a report but gave the decisive role to the USSR. ChatGPT "thinks" so too if asked a follow-up question. Meanwhile, GigaChat gave the most important role to US Lend-Lease.

Responsibility

None of the models wants to take responsibility for its answers. At a stage when large language models work as reference tools and ask you to double-check them, that seems fine.

But what will happen when these models are built into systems that book tickets and carry out transactions? Into autopilots and medical equipment? Into military hardware? Who will be responsible for these models' output? And how far will the paths of ChatGPT 4.0 and GigaChat have diverged by then?