Why it is time to make evaluation more relevant for Africa (3) – Emphasising Distinct African Values, Cultures, Institutions and Practices?

Just before I started to write this blog, I received the announcement of the 8th AfrEA International Conference that is scheduled to be held in Kampala, Uganada on 27-31 March 2017 on the theme “Evaluation of the SDGs: Opportunities and Challenges for Africa”. In this first announcement of the envisaged conference, the organisers stated that one of the primary goals of the conference is that it will act as “the foundation for promoting and advocating AfrEA’s ‘Made in Africa” approach’.

This is excellent news and will provide a platform for all evaluation specialists interested in evaluation in Africa to share and discuss their thoughts on this important topic and to attempt to reach agreement on possible strategies to take this initiative further in a systematic, coordinated and committed way. I therefore want to encourage all African evaluators to participate in this important and strategic event. In my first blog on this platform I summarized the reasons why the Western evaluation paradigm that has in the past influenced evaluation practices in Africa significantly, are still doing so. I motivated why this paradigm should perhaps be contextualized more effectively on the African continent, and adapted better to conditions that are in most respects quite different from the Western contexts within which evaluation developed into its current manifestation as a transdisciplinary global profession.

My second blog outlined the different stages through which programme evaluation developed in Africa, culminating in the establishment of AfrEA and numerous other African VOPES that all aim at professionalising evaluation in Africa, promoting Africa-rooted evaluation and building evaluation capacity for this purpose.

This third blog on the topic briefly assesses what the current Western and African evaluation paradigms comprise (if different attributes can in principle be distinguished), and which aspects of the Western paradigm, if any, need to be more effectively ‘Africanised’. I summarise below the main issues that I have already discussed at greater length at the 2015 SAMEA and AEA conferences.

The main arguments in favour of a more Africa-rooted evaluation paradigm centre around the fact that the most visible global evaluation approaches, models, theories, practices, specialists, scholars and practitioners that are currently operating in Africa are of Western origin. The most influential motivation for this to be changed, is that the current Western evaluation paradigm is not always optimal in Africa and needs to be contextualised to be more suitable to African conditions, cultures and institutions. Chilisa & Malunga, Ofir and the Bellagio Report on Evaluation and Development in Africa have all argued very strongly that prevailing African ontologies, epistemologies and methodologies have to be infused in a more holistic transdisciplinary manner into systematic programme evaluation on the continent. Chilisa and Malunga have concluded that there is a need for two main African transformations of current Western evaluation culture and practices. They are..

“…decolonizing and indigenizing evaluation … to recognize the adaptation of the accumulated Western theory and practice on evaluation to serve the needs of Africans” while the second is the development of a “’..relational evaluation branch’ (that) … draws from the concept of ‘wellness’ as personified in African greetings and the southern African concept of ‘I am because we are’. The wellness reflected in the relationship between people extends also to non-living things, emphasizing that evaluation from an African perspective should include a holistic approach that links an intervention to the sustainability of the ecosystem and environment around it”

Chilisa and Malunga adapted Alkin’s 2013 Evaluation Tree to fit African conditions better. One important implication of Chilisa and Malunga’s approach is that Africans should initiate their own more appropriate evaluation-related institutions, processes and cultures. Some examples of such developments that have already materialised, include the African Peer Review Mechanism (APRM), the Mo Ebrahim African Governance Index as well as in the African Sustainability Barometer and initiatives to clarify Africa’s role in pursuing the achievement of the Sustainable Development Goals (SDGs).

From this perspective, the main question to be answered is whether an ‘Africa-rooted’ evaluation paradigm would contain substantive differences from the prevailing ‘Western’ evaluation paradigm, or whether its purpose, focus, design and implementation would probably just be more sensitive to African cultural contexts and practices in order to achieve the most accurate and valid results.

Investigations into culturally sensitive evaluations in other continental contexts might provide a few guidelines to African evaluators in pondering these issues (eg Kwakami, Aton, Cram, Lai & Porima, AIHEC). This include evaluation design and implementation in the context of Native Americans, New Zealand Maoris, Australian Aborigines, South African Khoi-San, Brazilian, Indian and other indigenous cultural tribes and groups. There is further an active thought leadership stream in the American Evaluation Association that focuses on what they call ‘Culturally Competent Evaluation’ that take the salient differences in different cultural contexts into account in the design, implementation and management of evaluations in those contexts (eg La France & Nichols; Rog; Gervin). These schools of thought could provide important pointers to African evaluators in their quest to develop a more Africa-rooted evaluation paradigm for this continent.

The main questions that need to be concretely answered, probably include inter alia the following:

• Is it possible to identify Western, African, Asian and for that matter Latin-Caribbean evaluation approaches that differ fundamentally?, and
• what, if anything, should change in the prevailing western evaluation culture and practices for and in the African context?

At the moment there are no clear-cut answers to these questions. I suggest that one would be able to consider possible answers to these questions by distinguishing systematically the possible differences between a so-called Western and a more appropriate ‘African’ or other approach to different evaluation dimensions and activities.

One way to approach this issue could be to try to identify possible differences in Western and African evaluation planning approaches and practices in terms of their respective:

• values, principles & assumptions: reductionism vs holism?, ethics & professionalism vs realism & pragmatism?, efficiency , effectiveness, value for money & accountability vs loyalty, responsiveness, ubuntu, relationships, wellness, dignity, self-esteem, reconciliation, ancestors?, short term benefit vs long term empowerment, sustainability, resilience?
• purposes and foci: what, why, how, relevance, standards, nature of evidence – Western evaluation standards vs AfrEA evaluation guidelines?, and
• evaluation designs: qualitative, quantitative, mixed? level of rigour?, counterfactual, RCTs? power relationships? participatory and empowerment evaluations?

On the other hand one can consider possible differences in Western and African evaluation implementation and management strategies and practices in terms of their respective:

• data collection methods: rigorous statistical sampling vs culturally sensitive participatory processes? indigenous knowledge?
• data analysis methods: rigorous quantitative & qualitative analyses? causal attribution? generic indicators vs customised indicators for Africa? SDG Indicators? Indigenous methods? programme contribution?
• data presentation & communication strategies: written and oral? existing mass media vs oral & visual emphasis? different communication channels?
• evaluation use: improvement? accountability? empowerment?
• evaluation policy & regulatory frameworks: one-size-fits all? Legislation vs experimentation? context-sensitive customisation?, and
• evaluation capacity-building: education & training approaches – generic vs context sensitive content and facilitation?

If one can identify distinct substantive, institutional or procedural differences in the above activities between Western approaches and practices and those in the African context, the next question would be to what extent and how should evaluations in Africa then have to be changed in order to take these differences better into account. This is the main task for African evaluators who buy into the idea of a future African-rooted evaluation approach is desirable.

My next blog on this topic will deal with the question how a concrete roadmap for such a transition to a more Africa-rooted evaluation system could facilitate and fast-track achieving this goal.