Evaluation and the forbidden truth – Part 2

Walking the talk-Evaluation has become an integral part of programme and project design, implementation and management. This is a great achievement by those “slaving” courageous evaluators. Evaluators have become bolder and more scientific and evidence based in their search for the “Holy Grail” of evaluation – that is, wide understanding and acceptance of its true meaning, value and utility.

All seems well until one examines what happens to those “independent”, “evidence based”, “objective” or “impartial” evaluations.

Elaborate management responses, implementation plans, presentation to executive boards and other fora are steps in the right direction. However, this creates a false sense that action is being taken. The implementation of evaluation results and recommendations is lower than one would expect. This is true of most evaluations, but even more so in the case of evaluations of sensitive issues such as the drug programmes outlined in part 1 of this blog post.

Here is an example of a beautiful management response that in essence says we are going to do ‘nothing’: “The organization has spent the last two years reviewing our strategy and programmes and have already put in place the necessary changes. The evaluation recommendations are covered by these changes. This is not the time for more changes.”

Some years ago, an Under-Secretary-General concluded her stint at the UN with the following farewell message.

It hits at the core of some of the most basic challenges, showing how little prepared evaluators are for this job.

The “King Lion” phenomenon transcends international agencies, governments and civil society organization. The following is quoted from the letter.

“I am aware that some of you have challenges to the independence of your work; management in some cases would like to continue to maintain control over the ambit of your work. They want good news, not bad news. So when you have bad news, you learn to tell it in a clever way. Let me tell you a story.”

“There is in the old story of King Lion who calls all his subjects and asks them to tell him how his room smelled. Nobody dares to tell him anything, until the dog steps forward, sniffs the room and tells the King honestly that it smells bad. The King devours the dog for his insolence. The monkey then decides to be smarter and tells the King the room smells like roses. The King devours the monkey for his dishonesty and sycophancy. Lastly, with everyone in the room trembling with fear, the sly fox steps up and tells the King that he has had a cold for the past few days and cannot smell anything. The King rewards the fox by making him Prime Minister of his Kingdom.”

The USG continues “We in this room are NOT to be sly foxes. We are mandated to be dogs! So the question is – how do we survive as dogs when the King asks you how his room smells? To those of you who are facing challenges to your operational independence, and to your professional integrity, I would like to remind you of a quote by Dag Hammarskjold which I repeat now and again: ‘Never for the sake of peace and quiet, deny your own experience or conviction.’

“Would it not be sufficient to tell the King the truth if then the King cleans his den and all is roses? The problem is that we can never be assured that all Kings will respond correctly and clean the den, rather than devour the truth-speaking dog. As we can never be sure, it is better to put a system into practice, and develop a culture of truth-telling and transparency, than to hope and pray for a benevolent King each time.”

The USG did not tell us how “dogs should survive” when dealing with dog-devouring King Lions. She did not tell us what system or culture of truth-telling we should develop. Some years ago when I addressed the United Nations Evaluation Group (UNEG), I argued that evaluators are not trained in how to break bad news to executives. They are trained to gather evidence and they do this well. Then they go ahead and tell the truth. But they have never been trained on how to break bad news to the King and his Kingdom.

Evaluators should not be sly foxes, but how many “dogs” must die, especially when the sacrifices do not produce the desired results – that is, the King cleaning up his den?

I argue that, “dog training” is fundamental, but not enough to bring about changes in practices. Survival of evaluators is important, but at the core is how to make sure that the King Lions clean up their dens.

So, evaluators must be trained in how to deliver bad news in such a way that the King Lions clean their dens. But who will train the evaluators to do this? Or who will give them the power that allows them to do this?