VIDEO evaluation

There are as many ways in video evaluation as in any other branch of this sport…. You” find a good introduction in : http://www.evaluators5-0.net/index.php?page=Power%20of%20Images&id=94
by my good friend and colleague Marlen Arkesteijn.

What we intend to do with a small team of students and HA people is following, monitoring and start with evaluating the RDNA-activities @HA2010 by video, asking people’s cooperation for an interview and a short video (1 minute) after each event. We will then make short video compilations of what people have to say and place them on the website of HA2010, inviting others to comment… If your comments are interesting enough (!?!) you might be asked for an interview as well!
The final objective of this monitoring event is to try to find out what it does to people to be at events like this: what do they ‘learn’ – do they see practical implementation possibilities? What happens inside their heads (after the hangover…) and what happens later in their practice?
Don’t hesitate to comment and come up with tips: you’re most welcome!

The paradox of aid….

Some time ago I was in Nairobi for an evaluation of a global youth organization. I worked mainly with youth groups in Kibera and Mathare respectively, two ‘one million inhabitants slums’.

These hard working youngsters (all under 30) do hardly get any support from the international development organizations and not a shilling from their own government. Still they have over 50 members each, run small schools for orphans, organize ‘micro-credit groups’ themselves, have some money put aside for income generating loans (with 500 euro you can start a complete water distribution point, where 10 youngsters are busy with), run health advice for single mothers, etc. etc.

At night, back in my hotel I read in the papers that the minister of education has answered to the accusations of DFID from Britain that 30 million pounds have disappeared: He admits that it is at least 1 million pounds and that officials in his ministry have received the money for receipts manufactured by themselves. By the way: the minister stated that he would not resign, because he did not want to suffer for the misbehavior of his juniors… so much for the ministerial responsibility…

So the bilateral support for education in Kenya is now withheld by the UK government… Anyhow I saw with my own eyes that ‘education for all’ in Kenya (that is the programme Britain contributes to) means ‘income for the few’: no cent had been spent by the ministry of education nor by any civil society organizations to the 150 school kids in Mathare: shacks serve as schools (educated by volunteers from the community)…
And of course this kind of things do not only happen to British development aid, it happens all over the place.

What do I see in this example – one of many?
• In the first place it supports Patrick Chabals analysis (see ‘Southern Africa: Civil Society, Politics and donor strategies’(2009) to which I also contributed an article on Angola) that ‘power is about the control of resources’ – development aid is but one of the resources the political class controls – through the state…
• Further I saw once again that where the need is highest seemingly ‘chaotic’ and disorganized groups of volunteers take the lead and do something. Groups that are very seldom linked with registered charities and NGO’s (that can receive our international funds).
• In the third place it is clear that in the ‘aid arena’ the grassroots organizations are indeed in jeopardy (I quote Chabal again) because they do not meet our bureaucratic criteria – which paradoxically have come into life above all because of public outcries – specifically because of frauds like this story! So: because of the fact grassroots activists get no money from the government (that is subsidized by development agencies) aid on al large scale is frozen – so they will never get any money…. Do you still wonder why many people in sheer disbelief want to go and bring their own money to Africa? I can understand it better and better….

Total make-over

It’s not easy to evaluate an organisation that is in the middle of a ‘total make over’.

Still, thanks to the intense cooperation of many involved stakeholders, this became a very worthwhile operation, in the first place thanks to the transparent and co-creative atmosphere the evaluation was conducted in.
The South African Women’s fund in case was created in 1998 to ‘support local women, finding local solutions for local problems’ by providing grants to grassroots women, individuals and groups to capacitate them to come together and create their own economic and social justice and especially to reduce the gender based violence which is entrenched in these communities.
In the beginning of 2009, after the dismissal of then director, the board realised they were not on the ‘right track’ and decided to bring in a new director asking her to reform the organisation so that it would be ‘state-of-the-art’ again.
EVALUATION
The evaluation took place between October 2009 and February 2010 and its co-creative form fitted the organisation well: after some joint training with the grants’ staff they conducted interviews (in the Most Significant Change style), later scrutinised by myself as well, and these interviews were discussed with the whole staff and the board. The form of the rest of the evaluation was mainly topic-interviews and stakeholder meetings according to the ‘4th generation evaluation methodology’. I became rather a facilitator in the process, but nevertheless the final responsibility (e.g. for the conclusions and recommendations) is always mine. The evaluation was done by me and was immediately followed by a strategic planning, facilitated by Rosien Herweijer.
The main research question was: “How and to which extent has the fund reached its objectives in the past concerning the 4 areas of intervention: basic grants, seed funding to CBOs, women in leadership, discretionary grants, awards?”
The evaluation used 5 different ways (triangulation): a desk study, topic-interviews of stakeholders, gathering stories from grantees, participative observation and a total of 4 workshops with different stakeholders. This all took 4 working weeks (apart from the work done by the staff itself) and flowed into the strategic planning.
CLAIMS, CONCERNS & ISSUES
The so called Claims, Concerns and Issues (CC&I) were discussed in the final stakeholders’ meeting. The Claims, Concerns and Issues are the ‘informing principles’ during this whole evaluation. In them the evaluator initially constructs the good things of the organisation (the claims), but something is only a claim if there is consensus among the stakeholders (in the meetings representatives of staff, board and grantees, as well as sister organisations were present). Concerns are the things everybody agrees upon that need to be solved, ISSUES are all the other findings from the evaluation about which there is no consensus. CC&I were validated in meetings with the board, staff and other stakeholders of the organisation and later again in the strategic planning that followed the evaluation. Conclusions and recommendations iin the final report are then the evaluator’s interpretation of those CC&I.
STRATEGIC PLANNING
The organisation is at the moment of writing of this report halfway a strategic planning process, based on the findings and recommendations of this evaluation. The practical model used is a business plan, sorting out all customer segments and the value propositions made to them, rather than a theoretical model, or a too rigid model (like the log frame) that puts operations in a central place, rather than stakeholders’ groups. The basic business model has been built; the challenge is now converting it into a strategic plan (on paper). Future annual plans will fit into this strategic plan.
This way it became possible to adapt different assets from our toolbox to the needs of the organisation, and sofar you can say that right now, it was an experience that will later also allow the staff to build on it, not only in the form of the strategic planning, but also for future monitoring, etc.