Feeds:
Posts
Comments

Posts Tagged ‘report’

Emerging Opportunities: M&E in a tech-enabled world

Posted in accountability, development, evaluation, governance, ICT4D, ICTs, innovation, m4D, monitoring and evaluation, privacy, protection, technology, tagged , , , , , , , , on September 25, 2014| 4 Comments »

Today as we jump into the M&E Tech conference in DC (we’ll also have a Deep Dive on the same topic in NYC next week), I’m excited to share a report I’ve been working on for the past year or so with Michael Bamberger: Emerging Opportunities in a Tech-Enabled World.

The past few years have seen dramatic advances in the use of hand-held devices (phones and tablets) for program monitoring and for survey data collection. Progress has been slower with respect to the application of ICT-enabled devices for program evaluation, but this is clearly the next frontier.

In the paper, we review how ICT-enabled technologies are already being applied in program monitoring and in survey research. We also review areas where ICTs are starting to be applied in program evaluation and identify new areas in which new technologies can potentially be applied. The technologies discussed include hand-held devices for quantitative and qualitative data collection and analysis, data quality control, GPS and mapping devices, environmental monitoring, satellite imaging and big data.

While the technological advances and the rapidly falling costs of data collection and analysis are opening up exciting new opportunities for monitoring and evaluation, the paper also cautions that more attention should be paid to basic quality control questions that evaluators normally ask about representativity of data and selection bias, data quality and construct validity. The ability to use techniques such as crowd sourcing to generate information and feedback from tens of thousands of respondents has so fascinated researchers that concerns about the representativity or quality of the responses have received less attention than is the case with conventional instruments for data collection and analysis.

Some of the challenges include the potential for: selectivity bias and sample design, M&E processes being driven by the requirements of the technology and over-reliance on simple quantitative data, as well as low institutional capacity to introduce ICT and resistance to change, and issues of privacy.

None of this is intended to discourage the introduction of these technologies, as the authors fully recognize their huge potential. One of the most exciting areas concerns the promotion of a more equitable society through simple and cost-effective monitoring and evaluation systems that give voice to previously excluded sectors of the target populations; and that offer opportunities for promoting gender equality in access to information. The application of these technologies however needs to be on a sound methodological footing.

The last section of the paper offers some tips and ideas on how to integrate ICTs into M&E practice and potential pitfalls to avoid. Many of these were drawn from Salons and discussions with practitioners, given that there is little solid documentation or evidence related to the use of ICTs for M&E.

Download the full paper here! 

Read Full Post »

Riding the wave of disruption?

Posted in development, disaster and emergencies, disruption, failure, governance, politics, tagged , , , , , , , , , , on November 14, 2013| 4 Comments »

Back in May I participated in a discussion on if and how International Civil Society Organizations (ICSOs) are adapting to changes around them. Here’s my summary of the conversations: Can International Civil Society Organizations be nimble?

A final report from the meeting is ready (download Riding the Wave: A proposal for Boards and CEOs on how to prepare their organizations for disruptive change) and here’s a video summary:

I’m curious what other folks think about this topic and the analysis and recommendations in the report.

Read Full Post »

New report: Who Cares? Challenges and opportunities in communicating distant suffering

Posted in accountability, communications, development, disaster and emergencies, ICTs, mobile and technology, smart aid, social media, transparency, wait... what?, tagged , , , , , , , , , , , , , on June 28, 2012| 4 Comments »

Image captured from page 19 of the Polis report.

Who cares? Challenges and opportunities in communicating distant suffering: a view from the development and humanitarian sector, a study conducted by Polis, (the journalism think-tank within the Department of Media and Communications at the London School of Economics- LSE) with Plan UK, launched yesterday in London.
.
The study complements several pieces of research released over the past year or two on the UK public’s perception of foreign aid, development, giving, and the NGO sector; including Intermedia’s Building Support for International Development, the UK Public Opinion Monitor/Institute of Development Studies’ What Does the Public Think, Know and Do about Aid and Development? report, the Oxfam/Bond/DfID Finding Frames report and the Overseas Development Institute’s Understanding Public Attitudes to Aid and Development.

Whereas the other reports focus on various aspects of how the public sees charity, development and foreign aid, the Polis report gives a window into the debates and challenges of those working in advocacy, marketing, campaigning, fundraising and communication departments at INGOs. As my Plan UK colleague, Leigh Daynes, writes in the introduction:

‘…Our work to inform, to educate, to campaign for change and to recruit long-term supporters to fund change is valid. Often it is life-saving. Yet the least understood area of our work often is the impact of our communications on public understanding of and support for aid and development. The public are telling us they are saturated with suffering, that we are charming or disarming them into acts of compassion, and that we are abusing their emotions….

…Understanding the impact of the “lingua franca” of our industry matters because it has fuelled a template approach to the media reporting of suffering. It matters because the exponential growth in access to mobile and social media technology and platforms means we are no longer the de facto guardians we once were. And it matters because it speaks to the power between us and them, and you and me.’

As more and more research is done, the ways that the sector is shifting become clearer and I hope we will start to see some positive changes in the aid and development industry and the way it communicates with the general public.

Some angles and voices that could help round out the discussion are still missing, however. I would like to see research on the opinions of local and international staff managing programs on the ground. Most organizations have fierce internal discussions on how marketing and fundraising is done, as program staff often feel that some marketing and fundraising approaches are demeaning, disrespectful and undignified in their portrayal of program participants.  (For more on this, ask Talesfromthhood to share some of his blog posts with you, join some of the discussions on AidSource, or follow the #smartaid hash tag on Twitter). Some program staff worry that the long-term impact of media and fundraising shock tactics and overly simplistic messaging ‘cancels out’ the short-term gains achieved through program or emergency aid funding – a point that is raised in some of the research above – and that this contributes to two-dimensional views on aid and development and to the negative stereotypes about certain countries in Africa, Latin America and Asia that deter longer-term development and self-determination. It would be interesting to know more from this perspective.

What do donors want? Image captured from page 9 of the report.

At the same time, INGOs are hard-pressed to come up with different and viable ways of funding their work, so this type of campaigning and fundraising continues. Although I’m familiar with some of the challenges my marketing colleagues face, I was struck seeing the kinds of donor demands that INGOs are expected to meet listed in the report. I was reminded of something a marketing colleague once said: ‘Poverty porn. That’s a good term. It is a lot like real porn. People don’t like to publicly admit to watching and responding to it, but in private it’s another story. People say they don’t like poverty porn and sad, desperate stories, but when you look at the numbers, it’s what makes them reach into their pockets and give.’

I would also like to see some research with the subjects (unfortunately often presented as ‘objects’) of INGO aid and marketing materials.  How do program participants feel about how they are portrayed? What impact does it have on them and their own perspectives and self-determination, if any? Strong voices on this come from diaspora communities and from blogs such as Africa is a Country, Uganda 2012: Trending Our Own Stories, and great projects like My Africa Is; but I’d also like to see some in-depth research and objective focus groups that talk with those who are most often shown in INGO marketing. Are people aware of how they are being represented? Do they care? Many of us speculate about this when we bash ‘poverty porn’ but I’ve yet to see published research that involves actual ‘beneficiaries’ of INGO programs or people and communities appearing in INGO marketing and fundraising pieces in this discussion. (Maybe I’ve missed it – if you know of  any, please share!)

The ‘Who Cares’ report notes in the conclusion that:

  • transparency, accountability, ‘value for money,’ and impact are becoming more important to the donor public
  • public trust is a central concern for NGOs in their work and their communications
  • the sector needs to assume more collective responsibility for ethically appropriate portrayal of disaster victims (eg, in compliance with Article 10 of the Red Cross Red Crescent Code of Conduct)
  • new technologies and competition mean that fundraisers are seeking supporters outside of their traditional constituencies
  • traditional gatekeepers of aid are being challenged by new media’s ability to put donors and ‘beneficiaries’ in more direct contact

As INGO institutional leadership space opens up to more people from ‘the global South’, diaspora communities grow, social media allows for commercials and fundraising appeals to reach global audiences (including people in the countries where INGOs implement their programs), attention is paid to the ‘new bottom billion‘ and new fundraising mechanisms arise (for example, INGOs raising funds within countries where they are implementing programs), it will be interesting to see how the conversations and approaches shift and change — or if they remain the same with new actors taking on the same challenges.

Stay tuned for further research and reporting over the next couple of years from Polis and download the report here.

Read Full Post »

Tweaking: SMS violence reporting system in Benin

Posted in Benin, children, development, ICT4D, ICTs, mobile and technology, m4D, mapping, protection, VAC, youth, tagged , , , , , , , , , , , , , , , , on April 24, 2010| 6 Comments »

Henri from Plan Benin training on SMS reporting.

In February I was in Benin to support staff to pilot the idea of using SMS reporting (FrontlineSMS) and digital mapping (Ushahidi) to strengthen local and national systems for reporting, tracking and responding to violence against children.  We conducted 2 workshops in mid-February with youth leaders, frontline staff, community members, local authorities from the Center for Social Protection (CPS) and representatives from the Ministry of the Family to get things started.

For some more background, check out my previous posts on the Violence against Children (VAC) project, the questions we asked ourselves before getting started on the SMS and mapping initiative, and the February workshops in Benin and what we learned there.

Since February, staff in Benin have been following up with workshop participants and with local authorities and institutions, including: the Prefect, the Mayors, community supervisors, animators of the children’s/youth media clubs, headmasters and other school authorities and the CPS.  The youth in one community did a radio program about violence against children and talked about SMS reporting. They also designed an information sheet that’s been hung up all over the town to encourage the population to report cases of violence.

Henri, Plan Benin’s ICT Director who facilitated at the Benin workshops, went to Togo to replicate the training with staff and youth there.  He and Carmen, who manages the overall VAC project in Benin, have also been observing and collecting feedback on the system to see where it needs tweaking.  They have put a project plan together for the next 6 months or so.

Carmen the VAC project coordinator in Benin.

Observations that Henri and Carmen shared and some thoughts we have about resolving them:

Issue:

  • Most people call instead of sending SMS

Hmmm….

  • Why?  Habit?  Literacy?  Unclear indications of what to do or unclear expectations of what the system is for?  We need to find out more about this.  It would be good to know exactly what kind of volume we are talking about total in terms of SMS vs calls. (I will update this post when I find out.)
  • Should we start taking calls too then? And are there resources and capacity to manage calls in addition to FrontlineSMS (which is automated)? How are we linking with the Child Help Line in Benin?
  • Could both calls and SMS be administered in the Ushahidi system?  Eg., Just as an administrator needs to review any SMS’s that come into Ushahidi  before approving them, someone could be tasked with inputting information from a phone call into the Ushahidi back end to then trigger the rest of the process (verification, response, etc). And how would that impact on pulling data out of the system for decision making?  (See this post for more information on how the system is currently conceived)

Issue:

  • Some people are sending a re-call SMS (asking us to return the call)

Hmmm….

  • We need to find out why people are sending re-call messages instead of SMS’s.  Because in the current set-up, text messages are not free?  Literacy issues? Because our system looks like something else they’ve done where re-call was the norm?  Something else?
  • If it’s due to low literacy or language issues, how can we open the system to those who cannot read/write or who do not use French?
  • Plan Benin is discussing with the GSM provider to find a way to send back an automatic reply SMS informing people not to call but to send a message, and to take this opportunity to indicate in the message what is expected as information.  But if literacy/language is the issue, we will not have solved anything by doing this…. Sounds like we really need to make sure calling is an option, and that good integration with the national Child Help Line is a real priority.
  • Plan Benin is also negotiating getting a “green line” or free short code, so that might resolve part of this.

Issue:

  • Many people are not using the key word ‘HALTE’ (stop) at the beginning of the message, meaning that the commands don’t trigger the messages to automatically send the information to Ushahidi.  (In the current system, each SMS should include the key word ‘HALTE’.  This key word triggers a “thanks for your message” automatic response from FrontlineSMS, and the forwarding of the message to the Ushahidi back end for subsequent management and follow up by local authorities.)

Hmmm….

  • Staff noticed that most (but not all) of the messages without the key word ‘’HALTE’’ contained the word ‘enfant’ (child). Henri has added ‘enfant’ as a key word in addition to ‘HALTE’ — and says it is working fine.  So we will assess if this helps.
  • Another alternative would be to not use any keywords – we will need to look into whether we can set FrontlineSMS up so that any SMS that goes to that number gets auto forwarded to Ushahidi.

Early draft of a poster promoting violence reporting by SMS

Issue:

  • Most of the messages are too vague to find the place and the victim for responding (and people do not have GPS enabled phones).  We have suggested that an SMS report should contain certain information [HALTE+type of violence+where it’s happening (eg., school, home, etc)+village name+district+age+sex+name of child if known], but people don’t follow the suggested format.

Hmmm….

  • How can we simplify it or better explain the type of information that’s needed?  Something we need to dig deeper into and consult with users to figure out.  Carmen’s take is that we are at the beginning of the process and we need to be patient and sensitize a lot so that people get used to the idea and understand how things work.

Issue:

  • Compatible FLSMS phones and modems are very difficult to find.  We were only able to find one phone that was compatible in Benin (a used one) because newer phone models are not compatible and the modems we found refused to connect.

Hmmm….

  • We really need to get this resolved since the entire system in Benin rests on one phone. What if it stops working?  It’s really difficult to expand the project without a larger set of phone/modem options.  We’ll work with the FrontlineSMS forum or staff (both are always super helpful on this kind of thing) in the next couple weeks to figure out how to resolve the compatibility issues, because there are modems available in West Africa that should be compatible, but that we couldn’t get to function.

Issue:

  • We planned for community response teams to be able to subscribe to alerts on Ushahidi, so that when there is an incident reported in the zone where they work, they would be alerted by SMS and could set the follow up process in motion.  But we haven’t been able to get the alerts working on Ushahidi or set up email reporting there.

Hmmm….

  • We discussed with the Ushahidi team and the problem was that not all the strings of code in Ushahidi had been translated into French yet.  Thanks to @theresac and @penelopeinparis, who volunteered to translate a load of strings, we are getting everything into French, and Henry at Ushahidi is helping get alerts working.  We still need to finalize all the elements on our Ushahidi page however and get everything working.  We’d also like to customize our Ushahidi page to make it our own, similar to the customizing that Voices of Kibera has done with their Ushahidi instance.

Any additional thoughts or help on the above issues are most welcome!


As for next steps, Henri and Carmen shared their plans:

  • Present the system to political and administrative authorities, including: head of the Brigade for the Protection of Minors, juvenile judges, Ministry of the Family’s Director for Children and Adolescents, Director of Family Programs, Minister of ICTs, cabinet and authorities who regulate telecommunications, Ministry of the Interior and Public Security, National Assembly, mayors and prefects, schools and teacher-parent committees, community authorities, media
  • Train staff, government partners, school and parent committees, and local NGOs on the reporting system, including a 1-day workshop with all Plan staff and a 1 day workshop with local NGO partners, schools and government staff
  • Accompany child protection committees and organized youth groups to use the system.  This will be done by holding sessions with organized children and youth groups at village level to reinforce and raise awareness on the reporting system; training child protection committees to use the new reporting system; holding one day sessions each month with the village level child protection system staff to discuss follow up on reports that have come in, and installing FrontlineSMS in each local site and adding local focal points as Ushahidi administrators
  • Strengthen awareness in the public and with leaders to support violence reporting by developing a communications plan to generate awareness on the issue of violence, the importance of reporting, and the mechanisms to report via SMS; supporting youth to use arts and theater to raise awareness on the issue of violence against children; talking with religious leaders and village chiefs; creating television, radio, newspaper and web advertisements to reach the general public and decision makers
  • Secure a free short code (target:  by May)
  • Conduct a national level evaluation workshop with involved local and national actors (in 6 months)

As we move forward, more questions will surely come up and we’ll need to continually tweak things. But I feel that we’re off to a good start. The fact that people are calling in and SMS’ing in is a good sign already that the program has some potential, and that people are willing to report violence against children.

—————-

Related posts on Wait… What?

Breaking it down: violence against children

Fostering a new political consciousness on violence against children

Seven (or more) questions to ask before adding ICTs

Finding some ICT answers in Benin

Meeting in the middle

Read Full Post »