Annual Report 2007 Evaluation of Norwegian Development Cooperation

**"Continuous Tough Evaluation" from a "Strong Independent

Evaluation Department"**

This is what Erik Solheim, the Minister of the Environment

and International Development, expects of Norad's evaluation

work (The Trondheim daily Adresseavisa, 31 January

2008). By virtue of a clear mandate, the evaluation department

has wide authority to implement independent and

critical evaluations of how the Norwegian development

budget is spent. This remains the case whether the responsibility

for implementation is vested in the Ministry of Foreign

Affairs, in Norad, in the multilateral system, or in Norwegian

organisations.

We work systematically to cover all important parts of the

development budget. In 2007 we completed sector evaluation

of long-term aid in the petroleum area and for hydropower

development, and in 2008 we will evaluate the work

in the fisheries sector and assistance in preserving cultural

heritage. In 2007 we performed the first country evaluation

for a long time of the total Norwegian aid to one country,

in the form of the Zambia study. In 2008 we will get to

grips with aid to the Western Balkans, that is, the countries

of the former Yugoslavia.

We are systematising the evaluation of the work of Norwegian

non-government organisations (NGOs). In 2007 we

have looked at Guatemala and in 2008 we shall deal with

Uganda. The emphasis will be on results.

In the past year we have also performed two evaluations of

Norwegian humanitarian aid, one of the use of former military

trucks in emergency relief and the other of NOREPS -

the Norwegian Emergency Preparedness System. In 2008

and 2009 we will continue with evaluation of peacebuilding,

using as our point of departure the Guidance for evaluation

of peace-building and conflict prevention that the Evaluation

Department has helped to develop for OECD/DAC. Evaluation

of assistance in promoting human rights is next in line.

Are we then capable of living up to the Minister's expectations?

Everyone who has worked in evaluation knows how

difficult it is to give clear answers. We are facing challenges

on practically every front. Some of this is purely a matter of

methodology; as a rule, our data are unreliable. When aid is

harmonised with that from other donors it becomes difficult,

and often impossible, to evaluate the impact of Norwegian

funds and Norwegian input. Even as regards independence

and roles, however, we may encounter problems: the people

who know most about aid in a particular area often suffer

from a conflict of interest and cannot take part in the evaluations.

This means that the consultants must spend a lot of

time getting acquainted with what they are to evaluate. The

fact that foreign firms often get the assignments helps to

strengthen the independent evaluation, but it also creates

distance. Another factor is that the consultant industry is

often accused of toning down its criticism for fear of losing

future commissions. We do not believe this is the case, but

the perception may be real enough. The expertise of those

taking the assignments may also vary widely.

Our response to this is to be clear about roles and demand

a high quality. We are very strict on conflicts of interest and

regard it as our job both to encourage critical independence

and to protect the consultant against what can easily be perceived

as pressure from the objects of the evaluation. We are

laying increasing emphasis on having the results of the aid

documented. We think that we have largely succeeded in this,

as in the Zambia study, the Guatemala evaluation and in the

evaluation of the hydropower aid. In the future, however, we

shall be looking more closely at the use of money and at

cost-effectiveness than we have so far managed to do.

The Zambia evaluation is one example of a "tough" evaluation,

in which fundamental questions were asked, and in

which good results were obtained as regards institutionbuilding,

at the same time as we found reason to enquire

whether the aid was sufficiently poverty-oriented. There

may be more such evaluation projects in the future, where

we also make use of other approaches to evaluation than

those we have traditionally employed.