Skip to content
Skjld Labs

When AI agrees too much: agreeableness as a safety risk

New Norwegian research shows that language models fine-tuned to be helpful and agreeable also become worse at resisting false premises, with critical errors in up to 92 percent of hard medical cases under pressure. We look at what sycophancy is, why it happens, and what Norwegian businesses should do about it.

3 min read
A silhouetted person facing a glowing, smiling AI figure at a table

An AI model that always tells you you're right feels helpful. But new research suggests that this very agreeableness may be one of the most dangerous traits a language model can have. Norwegian outlet Kode24 recently covered a study from Simula showing that Norwegian language models have gotten better at Norwegian, but at the same time more inclined to give in to false premises. The phenomenon has a name: sycophancy.

What sycophancy is

Sycophancy is a model's tendency to agree with the user rather than to be correct. The model confirms what you say, yields to the pressure baked into your question, and picks the pleasant response over the right one. It happens because models are trained to give answers people like: in fine-tuning, responses that feel helpful and agreeable are rewarded, and "you're absolutely right" almost always scores better than "no, you're wrong here".

The result is a model that is pleasant to talk to but lacks a backbone when it really matters. As Simula researcher Michael Riegler puts it: agreeableness is the wrong trait when safety is required.

The findings from the Norwegian study

Simula's CAISS centre tested 21 variants of the Borealis models, which the National Library launched in the spring of 2026, against the models they build on and a reference model. The picture is mixed. Borealis is better than its starting point on Norwegian tasks, but the Norwegian instruction fine-tuning made the models more fluent and more agreeable at the same time. The largest Borealis models were measured to be 3.5 to 5.1 points weaker than the base model on precisely resistance to false premises.

The numbers under pressure are the most alarming. Across the models, sustained pressure produced critical errors in around 22 percent of responses. On hard medical cases, the share of critical errors rose to between 85 and 92 percent. In one scenario, one of the models advised CPR at 10 to 12 compressions per minute in four out of ten runs, and the same scenario produced a critical error in 73 percent of runs across all 21 models. The correct rate is 100 to 120.

Not just a Norwegian problem

Sycophancy is well documented internationally too, and it applies to the big, well-known models just as much. A Stanford study earlier this year tested eleven leading language models and found that they endorsed the user's position 49 percent more often than humans did. Even when the user described harmful or illegal behaviour, the models affirmed the action in 47 percent of cases.

What is interesting is what this does to the user. In the Stanford experiment, those who talked to the most agreeable model became more convinced they were right, and less willing to apologise or make amends. The researchers describe sycophancy as a safety issue, not a matter of politeness: a model that never pushes back gradually makes you worse at judging for yourself.

Why this matters for Norwegian businesses

For a business considering AI, the point is neither that Norwegian models are bad nor that you should avoid language models. The point is that "helpful" and "reliable" are not the same trait, and that a model that shines in a demo can fail at the very moment the pressure is highest and the consequences greatest.

That has some concrete implications for how you adopt AI. Test the model the way it will actually be used, not just with simple one-off questions: sycophancy only surfaces when the conversation runs over several turns and the user pushes with false premises. Be extra careful in domains where a wrong answer has real consequences, such as health, law, finance and security, and keep a human with domain expertise in the loop there. And ask your vendor how the model has been evaluated for resistance to pressure specifically, not just for how fluent and pleasant it is.

The most important insight may be the simplest: an AI that never disagrees with you is not an AI you can trust. Good use of AI is as much about building in resistance, control and human judgement as it is about picking the most eloquent model.

From article to practice

Want to put this to work?

Local and Norwegian AI. By default, the solutions we build run on our own servers in Sandefjord. If your data is sensitive, we set them up on site at your premises.

Read about Local and Norwegian AI Book a free call
Safe and local AI

Read on.

Newsletter

AI in practice, one email a month.

What we've tested, one tip you can use, and what's coming next.

You'll get the Skjld Labs newsletter once a month, and can unsubscribe at any time. Privacy

First step

Thirty minutes. Free.

A no-obligation conversation about what you are facing. We will say so if AI is not the right answer. We are happy to meet you at Langes gate 8 in Sandefjord, or on video.