Feeds:
Posts
Comments

Posts Tagged ‘ICT’

I attended the African Evaluators’ Conference (AfrEA) in Cameroon last week as part of the Technology and Evaluation strand organized by Pact with financial support from The Rockefeller Foundation’s Evaluation Office and The MasterCard Foundation. The strand was a fantastic opportunity for learning, sharing and understanding more about the context, possibilities and realities of using ICTs in monitoring and evaluation (M&E). We heard from a variety of evaluators, development practitioners, researchers, tool-developers, donors, and private sector and government folks. Judging by the well-attended sessions, there is a huge amount of interest in ICTs and M&E.

Rather than repeat what’s I’ve written in other posts (see links at the bottom), I’ll focus here on some of the more relevant, interesting, and/or new information from the AfrEA discussions. This first post will go into institutional issues and the ‘field’ of ICTs and M&E. A second post will talk about design and operational tips I learned /was reminded of at AfrEA.

1) We tend to get stuck on data collection –Like other areas (I’m looking at you, Open Data) conversations tend to revolve around collecting data. We need to get beyond that and think more about why we are collecting data and what we are going to do with it (and do we really need all this data?). The evaluation field also needs to explore all the other ways it could be using ICTs for M&E, going beyond mobile phones and surveys. Collecting data is clearly a necessary part of M&E, but those data still need to be analyzed. As a participant from a data visualization firm said, there are so many ways you can use ICTs – they help you make sense of things, you can tag sentiment, you can visualize data and make data-based decisions. Others mentioned that ICTs can help us to share data with various stakeholders, improve sampling in RCTs (Randomized Control Trials), conduct quality checks on massive data sets, and manage staff who are working on data collection. Using big data, we can do analyses we never could have imagined before. We can open and share our data, and stop collecting the same data from the same people multiple times. We can use ICTs to share back what we’ve learned with evaluation stakeholders, governments, the public, and donors. The range of uses of ICTs is huge, yet the discussion tends to get stuck on mobile surveys and data collection, and we need to start thinking beyond that.

2) ICTs are changing how programs are implemented and how M&E is done — When a program already uses ICTs, data collection can be built in through the digital device itself (e.g., tracking user behavior, cookies, and via tests and quizzes), as one evaluator working on tech and education programs noted. As more programs integrate digital tools, it may become easier to collect monitoring and evaluation data with less effort. Along those lines, an evaluator looking at a large-scale mobile-based agricultural information system asked about approaches to conducting M&E that do not rely on enumerators and traditional M&E approaches. In his program, because the farmers who signed up for the mobile information service do not live in the same geographical community, traditional M&E approaches do not seem plausible and ICT-based approaches look like a logical answer. There is little documentation within the international development evaluation community, however, on how an evaluator might design an evaluation in this type of a situation. (I am guessing there may be some insights from market research and possibly from the transparency and accountability sectors, and among people working on “feedback loops”).

3) Moving beyond one-off efforts — Some people noted that mobile data gathering is still done mostly at the project level. Efforts tend to be short-term and one-off. The data collected is not well-integrated into management information systems or national level processes. (Here we may reference the infamous map of mHealth pilots in Uganda, and note the possibility of ICT-enabled M&E in other sectors going this same route). Numerous small pilots may be problematic if the goal is to institutionalize mobile data gathering into M&E at the wider level and do a better job of supporting and strengthening large-scale systems.

4) Sometimes ICTs are not the answer, even if you want them to be – One presenter (who considered himself a tech enthusiast) went into careful detail about his organization’s process of deciding not to use tablets for a complex evaluation across 4 countries with multiple indicators. In the end, the evaluation itself was too complex, and the team was not able to find the right tool for the job. The organization looked at simple, mid-range and highly complex applications and tools and after testing them all, opted out. Each possible tool presented a set of challenges that meant the tool was not a vast improvement over paper-based data collection, and the up-front costs and training were too expensive and lengthy to make the switch to digital tools worthwhile. In addition, the team felt that face-to-face dynamics in the community and having access to notes and written observations in the margins of a paper survey would enable them to conduct a better evaluation. Some tablets are beginning to enable more interactivity and better design for surveys, but not yet in a way that made them a viable option for this evaluation. I liked how the organization went through a very thorough and in-depth process to make this decision.

Other colleagues also commented that the tech tools are still not quite ‘there’ yet for M&E. Even top of the line business solutions are generally found to be somewhat clunky. Million dollar models are not relevant for environments that development evaluators are working in; in addition to their high cost, they often have too many features or require too much training. There are some excellent mid-range tools that are designed for the environment, but many lack vital features such as availability in multiple languages. Simple tools that are more easily accessible and understandable without a lot of training are not sophisticated enough to conduct a large-scale data collection exercise. One person I talked with suggested that the private sector will eventually develop appropriate tools, and the not-for-profit sector will then adopt them. She felt that those of us who are interested in ICTs in M&E are slightly ahead of the curve and need to wait a few years until the tools are more widespread and common. Many people attending the Tech and M&E sessions at AfrEA made the point that use of ICTs in M&E would get easier and cheaper as the field develops, tools get more advanced/appropriate/user-friendly and widely tested, and networks/ platforms/ infrastructure improves in less-connected rural areas.

5) Need for documentation, evaluation and training on use of ICTs in M&E – Some evaluators felt that ICTs are only suitable for routine data collection as part of an ongoing program, but not good for large-scale evaluations. Others pointed out that the notions of ‘ICT for M&E’ and ‘mobile data collection/mobile surveys’ are often used interchangeably, and evaluation practitioners need to look at the multiple ways that ICTs can be used in the wider field of M&E. ICTs are not just useful for moving from paper surveys to mobile data gathering. An evaluator working on a number of RCTs mentioned that his group relies on ICTs for improving samples, reducing bias, and automatically checking data quality.

There was general agreement that M&E practitioners need resources, opportunities for more discussion, and capacity strengthening on the multiple ways that ICTs may be able to support M&E. One evaluator noted that civil society organizations have a tendency to rush into things, hit a brick wall, and then cross their arms and say, “well, this doesn’t work” (in this case, ICTs for M&E). With training and capacity, and as more experience and documentation is gained, he considered that ICTs could have a huge role in making M&E more efficient and effective.

One evaluator, however, questioned whether having better, cheaper, higher quality data is actually leading to better decisions and outcomes. Another evaluator asked for more evidence of what works, when, with whom and under what circumstances so that evaluators could make better decisions around use of ICTs in M&E. Some felt that a decision tree or list of considerations or key questions to think through when integrating ICTs into M&E would be helpful for practitioners. In general, it was agreed that ICTs can help overcome some of our old challenges, but that they inevitably bring new challenges. Rather than shy away from using ICTs, we should try to understand these new challenges and find ways to overcome/work around them. Though the mHealth field has done quite a bit of useful research, and documentation on digital data collection is growing, use of ICTs is still relatively unexplored in the wider evaluation space.

6) There is no simple answer. One of my takeaways from all the sessions was that many M&E specialists are carefully considering options, and thinking quite a lot about which ICTs for what, whom, when and where rather than deciding from the start that ICTs are ‘good and beneficial’ or ‘bad and not worth considering.’ This is really encouraging, and to be expected of a thoughtful group like this. I hope to participate in more discussions of this nature that dig into the nuances of introducing ICTs into M&E.

Some new resources and forthcoming documentation may help to further set the stage for better learning and application of ICTs in the M&E process. Pact has just released their Mobile Technology Toolkit, and Michael Bamberger and I are finishing up a paper on ICT-enabled M&E that might help provide a starting point and possible framework to move things forward. The “field” of ICTs in M&E is quite broad, however, and there are many ways to slice the cake. Here is the list of toolkits, blog posts and other links that we compiled for AfrEA – please add any that you think are missing!

(Part 2 of this post)

Previous posts on ICTs and M&E:

Advertisements

Read Full Post »

This is a guest post by Daniella Ben-Attar (@dbenattar) who consults for international development agencies, NGOs and corporations on areas relating to youth participation, governance, municipal capacity building, ICT4D and peace building.

by Daniella Ben-Attar

Youth in Mali with local authorities.

Youth in Mali with local authorities.

ICTs are increasingly being looked to as holding great promise for improving participatory governance and citizen engagement. Mobile phones have been a game-changer in this sphere, with nearly seven billion mobile-cellular subscriptions worldwide, including 89% penetration in the developing world. Youth are at the center of these developments, both as drivers and consumers of technological innovation.  This is particularly true in developing countries where the young generation is leading the way in the usage of technology to overcome social, political and economic exclusion to begin driving positive change in their communities. The largest cohort in history, youth aged 15-24 number more than 1.2 billion worldwide, with an estimated 87% living in developing countries.  They are almost twice as networked as the global population as a whole, with the ICT age gap more pronounced in least developed countries where young people are often three times more likely to be online than the general population.

The combination of the “youth bulge” and “mobile miracle” has great potential to enable new responses to the longstanding challenge of youth engagement in governance across the developing world. Young citizens are utilizing simple mobile technology to innovate new platforms, tools and mechanisms aiming to amplify their voices and influence government. Youth are being proactive to play a greater role in governance through mobile-based communication avenues, user-generated information, tools tracking government accountability, anti-corruption platforms, crowd-sourcing and more. This is a dramatic shift from the days when the only way to gain the attention of a government official was through slow and cumbersome bureaucratic processes and official meetings in government offices.

A Growing Youth-Local Government Disconnect

Ironically, the impact of these efforts appears to be more pronounced at the national level than at the local level of government. Indeed, ICTs seem to be strengthening communications between youth and central government instead of enhancing connections with the closest level of governance where young citizens can be resources for community development. Applications and innovations in cooperation with government that address local issues have largely been the product of national government bodies. Most youth-led initiatives have not been successful in securing local government partnership, limiting impact. A communications gap has widened between young citizens and their local governments, which are often staffed by individuals with far less digital experience than their youthful constituents. As a result, youth and their local leaders often seem to be speaking in different languages through different media.  Local government deficits in capacity and resources continue to exist as barriers, as well as the need for sensitization to youth engagement as a priority outcome of adopting and shaping ICT-enabled practices.

Most young people using technology as a way to influence governance will tell you a similar story. When expressing themselves through social media outlets and ICT-enabled mechanisms, it is usually the national political figures that are more attuned and responsive. Local leaders are far behind their national counterparts in ICT capacity and usage. National ministers and officials often use Twitter accounts, blogs, SMS and websites to engage with their citizens, who by default are largely young. While this is a positive development, it also elicits frustration from young people who feel that their voices are ignored or unheard by elder leaders at the local level where chances are greatest for tangible impact in their day-to-day lives.

President Kagame of Rwanda is a stark example.  Youth have described how the president directly interacted with young citizens via Twitter and addressed concerns relating to many issues, from police violence towards youth to business ideas for urban tourism.  No such possibilities existed for these same youth to approach the local authority with these locally-based needs.  Even more significant, Kagame merged the national ministries of Youth and ICT in 2012 and appointed a Minister of Youth and ICT.  This is a groundbreaking move both in terms of ICT and youth, with youth ministries commonly grouped with sports or culture. However, these extraordinary national developments are not reflected in the policy and practice of local government in Rwanda.

Digital mapping initiatives have been in the spotlight as a new youth-driven tool drawing attention to local issues often overlooked by government officials.  While communities are benefitting from these processes, youth leaders report that these maps often do not gain the attention of city hall. For example, Kenyan NGO Map Kibera has seen its maps utilized by national ministry committees, better equipped with the capacity and mindset to absorb digital data, while city council has not been responsive to ICT-based approaches. Young leaders in Kandy City, Sri Lanka are working to bridge the “youth-local government ICT gap” which they have identified as a major barrier in engaging youth in local development. These young leaders are training municipal officials in computer skills and creating new ICT platforms for citizen-local government interaction as part of a UN-HABITAT supported youth-led training and education program run by YES – City of Youth.

Building Local Government Capacity for ICT & Youth Engagement

Partnership with local government is viewed by stakeholders as a key missing ingredient in enabling governance technology applications to have tangible results at the community level. The importance of “closing the engagement loop” and early local government buy-in is emphasized time and again by stakeholders in the field as a vital lesson learned through pilot programs. Youth organizations like Youth Agenda and Sisi ni Amani have achieved successful governance results by engaging local leaders as partners from the preliminary stages, highlighting the benefits they can gain through mobile solutions that increase civic engagement, enhance service delivery, fight corruption and bridge between local government and citizens.

Bridging the youth-local government gap will require sensitizing local officials and encouraging them to see the advantages of “listening” to youth ICT platforms, to bring them to where the majority of youth are voicing their opinions, and enable them to take responsive actions. National governments should be encouraged to help local governments be better equipped to address youthful concerns at the local level through capacity building for both youth engagement and ICT4G.  This can be supported by integrating local ICT components in national ICT plans, or increased “decentralization” and integration of both youth and ICT strategies, bolstered by budgetary allocations and devolution of authority. When seeking to utilize ICT to deliver positive governance outcomes for young people, “local gov” must be part of the “ICT4Gov” equation.

This blog post draws on findings from a UN-HABITAT Report entitled “ICT, Urban Governance and Youth” co-authored by Daniella Ben-Attar and Tim Campbell.

Read Full Post »

Our February 6th Technology Salon in New York City focused on the organizational challenges that development organizations face when trying to innovate or integrate ICTs into their programs and operations. We looked at the idea of “innovation” and different ways to approach it. We asked what “innovation” really means and why “technology” and “innovation” seem to always be used interchangeably. We shared ideas, challenges and good practice around supporting and encouraging staff, managers, and donors to experiment with new and better ways of doing things.

A huge thank you to Somto Fab-Ukozor and Rachana Kumar for their collaboration on writing the summary below!

Mika

Mika Valitalo, Plan Finland. (Photo by Somto Fab-Ukozor)

Our lead discussants were Jessica Heinzelman, DAI’s senior ICT specialist; Chris Fabian, UNICEF’s advisor to the Executive Director on innovation and co-lead of UNICEF’s innovation lab; and Mika Valitalo, Plan Finland’s program manager for ICT4D.

What is innovation?

Different organizations bring in different ideas and definitions of innovation. Is innovation always synonymous with technology? Does it always require technology? For some organizations, “innovation” means doing things faster, better and differently in a way that adds value and has a concrete impact.

One discussant noted that innovation is not necessarily disruptive in nature; it can be categorized into 3 main forms:

  • a totally new context, new problem, new solution
  •  an existing solution that is improved
  •  an existing solution that is adapted to a new context, country or sector

Another lead discussant pointed out that innovation is not necessarily something brand new; it can be something that existed but that is used in a different way or simply different processes or ways of thinking, and innovation does not have to be technology. The concept of innovation is often misunderstood, he said, because “someone can come up with 10 crappy ideas that are new but that does not make them innovative or useful.” He also cautioned that innovation should not only be about replication and scale, yet donors sometimes decide that an idea is innovative and encourage organizations to replicate the idea, without ensuring that it is having a real or relevant impact across different local contexts.

One discussant disagreed and said that there’s no innovation without technology; for example, 60% of kids are stunting in one of the greenest areas in the world because of lack to electrical grid; the provision of electricity is technology. Without the electrical grid, the country will never reach any of its developmental goals. Technology enables the work to happen. A different viewpoint, as another discussant explained, was that the application of the technology is the innovative part, not the technology itself.

What fuels innovation?

A key part of the Salon discussion focused on whether having dedicated resources fueled innovation, or whether the presence of challenges and constraints forces innovation. Some Salon participants felt that when people are faced with challenges such as less time, fewer resources, no office space, etc., they may find themselves being more innovative in order to overcome constraints. Others found that staff often use the excuse of not having time and resources as a reason for not innovating or thinking outside the box. Some felt that innovation is difficult to achieve within large bureaucratic institutions due to their risk averse cultures, whereas others felt that one of the benefits of large-scale organizations is having resources to innovate and then test and scale innovations. Participants did agree that regardless of the outside setting, some people are more inclined to be innovative – these people are easy to identify almost everywhere, as they are always coming up with new ideas and trying/testing things out. The key is to find a way for organizational structures to support and reward innovators.

Encouraging innovation within large development organizations

Different organizations approach the innovation question in different ways. One discussant said that at his organization, the innovation team spends 60% of its time working on problems the organization is facing at the moment; 20% of its time looking towards the future (a 3-5 year horizon) for ideas that have an immediate direct impact on its work; and 20% of its time on organizational redesign, in other words, how to work with users to create solutions that are not top down and that take advantage of the existing ecosystem. His innovations team is only interested in finding/creating innovations that could reach very large scale, such as 10,000,000 people or more.

The innovation team created some guidelines for staff and allies with tips on how to defend one’s existence as someone working on innovation.  The guide addresses questions like: Why innovation?  Is it valuable to have an innovation unit? If so, why? If so and why, then prove it. Working on these questions led the innovation unit to develop metrics for innovation to justify staff positions focused on innovation. These guidelines can help people at other organizations who are trying something new to have a reference point; they allow innovation teams to say “such-and-such organization is doing this, so we can do it too.”

Metrics for innovation

Having a set of metrics can help innovation labs, teams or persons charged with organizational innovation to measure whether they are actually achieving their goals, too. One organization defined the following metrics:

  • permission to fail or fail cheaply without fear
  • working with heterogeneous groups
  •  sharing knowledge across countries and contexts

Working across organizational boundaries without “soul crushing bureaucracy” and having the real ability to work horizontally is one key to achieving these metrics.

Decentralizing the innovation function

Another lead discussant described the institutional changes and underlying understanding of people needed to improve and support innovation:

  • Identify the real incentives that someone has – individual or project – and the disincentives to innovating. It is important to look underneath the excuses people come up with such as time constraints and additional work, and find out what is driving them.
  • Hire realistic optimists – Sometimes in the ICT4D space, people gloss over the challenges and promote the technology. It is important to hire people who are grounded and have a good analytical sense, and who can think beyond gadgets and hype.
  • Building and sharing expertise within the organization – Creating a champions group of mid-to entry-level professionals within the organization, who understand the power that new technology has, is another way to make innovation and ICT4D spread. Rather than keep the expertise isolated within a specialist unit, finding younger people who are hungry for knowledge and who see this kind of work as a way to help further their career and set themselves apart from their colleagues can help. Then the “innovation team” can provide them with support and guidance. Participatory workshops on new tools and approaches can be organized where these innovation champions are tasked to research and explore something and then present it. Equipped with tools and trainings, they will be able to better identify opportunities for innovation.
  • Getting innovation into the plan early and working with those who are putting proposals and RFPs together to make sure that it is part of the metrics being measured from the beginning. It’s hard to add new elements into the program later because people will perceive it as additional work.

One Salon participant said that her organization disconnected “innovation” from its other programs so that space for trying new things would be made, and the fear of failing would be reduced or “offloaded” to the innovation team. In this case, the unit is funded through private sources which support it to experiment. It still has to struggle for its existence and show the impact and value of either failure or success.

Ideas for taking innovation and ICT4D forward

Some ideas for moving ahead included:

  1. Flexibility in program planning– In reality, most times during program implementation the plan changes and we have to figure out how to cope with it. The solution lies in the ability to quietly promote innovation and to influence donor organizations to embrace more flexible implementation.
  2. Integrating User-Centered-Design – Ethnographic research can help to better understand how people use technology locally and what its meaning is. It also helps identify existing patterns and ways of doing things that could be enhanced or shared with other communities if they are working well. Agile methodology from the software world can be pulled into development programs in order to end the top-down approach of solving problems from afar and having everything cooked up from the start. Rather, focusing on small iterations and the impact of the deliverables can be a better approach.
  3. Collaboration with Universities – Universities can be great places for working on and trying out  new ideas. Links with universities can be used as ways to find solutions, but even moreso to “change the proteins” inside of a traditional organization.  Collaboration among staff and students provides opportunities for staff to learn how to think about things differently and for students to understand real-world challenges in development agencies.
  4. Bridging the gap – Involving educators, health experts, child protection specialists and others who are not very interested in gadgets can bring about strong understanding of the real needs. Then connecting them with “techies” and ICTs in plain language and asking them to relate their own use of tech (they probably all use mobile phones in their personal lives, for example) to the ways that community members use tech can help to bring about solid, practical, sustainable and locally driven solutions.
  5. Provide a safe environment – Many humans are innovative by nature, said one discussant. Hierarchies and organizational processes are often what prevent people from doing new things. Giving feedback and psychological support can help those who are innovative to flourish within a difficult environment.
  6. The interdisciplinary approach – One Salon participant said that his organization had started to work with some senior staff to think and structure data in a way that would help them understand their challenges and programs better in order to innovate. This makes people more comfortable, and working across different teams with a variety of people and skill sets can help new ideas and solutions to bubble up.
  7. Information intermediaries – Infomediaries working at various levels can help connect people with technology, conduct training, and ensure that staff can acquire skills to use the technology themselves and in programs.
  8. Open source – Making project documents, budgets, concepts, “open” online can make them more accessible and  help  enable sustainable projects and prevent issues and costs associated with proprietary tools, applications and content.
  9. Younger management – There’s an age differential between the people who lead most large organizations and large-scale projects and those who are more interested in technology. One participant suggested it would be important to get younger people into positions where they can make contributions of ideas and decisions without being blocked by higher level people that may be “past their innovation prime.” Another solution may be to hire more experienced people but to ensure that they are open to working with  younger people who bring in new ideas. (Some Salon participants, however, felt that age has nothing to do with innovation, and that it is more related to personality types and organizational environments).

For  additional resources on the Salon topic, look here – and add your resources as well.

Salons are held under Chatham House Rule, therefore no attribution has been made in this post. Many thanks to our lead discussants and to ThoughtWorks for hosting and providing breakfast.

If you’d like to attend future Salons, sign up here!

Read Full Post »

Screen Shot 2013-11-23 at 6.14.40 PM

Migration has been a part of the human experience since the dawn of time, and populations have always moved in search of resources and better conditions. Today, unaccompanied children and youth are an integral part of national and global migration patterns, often leaving their place of origin due to violence, conflict, abuse, or other rights violations, or simply to seek better opportunities for themselves.

It is estimated that 33 million (or some 16 percent) of the total migrant population today is younger than age 
20. Child and adolescent migrants make up a significant proportion of the total population of migrants in Africa (28 percent), Asia (21 percent), Oceania (11 percent), Europe (11 percent), and the Americas (10 percent).

The issue of migration is central to the current political debate as well as to the development discussion, especially in conversations about the “post 2015” agenda. Though many organizations are working to improve children’s well-being in their home communities, prevention work with children and youth is not likely to end migration. Civil society organizations, together with children and youth, government, community members, and other stakeholders can help make migration safer and more productive for those young people who do end up on the move.

As the debate around migration rages, access to and use of ICTs is expanding exponentially around the globe. For this reason Plan International USA and the Oak Foundation felt it was an opportune time to take stock of the ways that ICTs are being used in the child and youth migration process.

Our new report, “Modern Mobility: the role of ICTs in child and youth migration” takes a look at:

  • how children and youth are using ICTs to prepare for migration; to guide and facilitate their journey; to keep in touch with families; to connect with opportunities for support and work; and to cope with integration, forced repatriation or continued movement; and
  • how civil society organizations are using ICTs to facilitate and manage their work; to support children and youth on the move; and to communicate and advocate for the rights of child and youth migrants.

In the Modern Mobility paper, we identify and provide examples of three core ways that child and youth migrants are using new ICTs during the different phases of the migration process:

  1. for communicating and connecting with families and friends
  2. for accessing information
  3. for accessing services

We then outline seven areas where we found CSOs are using ICTs in their work with child and youth migrants, and we offer some examples:

Ways that CSOs are using ICTs in their work with child and youth migrants.

Ways that CSOs are using ICTs in their work with child and youth migrants.

Though we were able to identify some major trends in how children and youth themselves use ICTs and how organizations are experimenting with ICTs in programming, we found little information on the impact that ICTs and ICT-enabled programs and services have on migrating children and youth, whether positive or negative. Most CSO practitioners that we talked with said that they had very little awareness of how other organizations or initiatives similar to their own were using ICTs. Most also said they did not know where to find orientation or guidance on good practice in the use of ICTs in child-centered programming, ICTs in protection work (aside from protecting children from online risks), or use of ICTs in work with children and young people at various stages of migration. Most CSO practitioners we spoke with were interested in learning more, sharing experiences, and improving their capacities to use ICTs in their work.

Based on Plan Finland’s “ICT-Enabled Development Guide” (authored by Hannah Beardon), the Modern Mobility report provides CSOs with a checklist to support thinking around the strategic use of ICTs in general.

ICT-enabled development checklist developed by Hannah Beardon for Plan International.

ICT-enabled development checklist developed by Hannah Beardon for Plan International.

We also offer a list of key considerations for practitioners who wish to incorporate new technologies into their work, including core questions to ask about access, age, capacity, conflict, connectivity, cost, disability, economic status, electricity, existing information ecosystems, gender, information literacy, language, literacy, power, protection, privacy, sustainability, and user-involvement.

Our recommendation for taking this area forward is to develop greater awareness and capacity among CSOs regarding the potential uses and risks of ICTs in work with children and youth on the move by:

  1. Establishing an active community of practice on ICTs and children and youth on the move.
  2. Mapping and sharing existing projects and programs.
  3. Creating a guide or toolbox on good practice for ICTs in work with children and youth on the move.
  4. Further providing guidance on how ICTs can help “normal” programs to reach out to and include children and youth on the move.
  5. Further documentation and development of an evidence base.
  6. Sharing and distributing this report for discussion and action.

Download the Modern Mobility report here.

We’d love comments and feedback, and information about examples or documentation/evidence that we did not come across while writing the report!

Read Full Post »

At the November 8th Technology Salon in New York City, we looked at the role of ICTs in communication for development (C4D) initiatives with marginalized adolescent girls. Lead discussants Kerida McDonald and Katarzyna Pawelczyk discussed recent UNICEF reports related to the topic, and John Zoltner spoke about FHI360’s C4D work in practice.

To begin, it was pointed out that C4D is not donor communications or marketing. It is the use of communication approaches and methodologies to achieve influence at various levels —  e.g., family, institutional and policy —  to change behavior and social norms. C4D is one approach that is being used to address the root causes of gender inequality and exclusion.

Screen Shot 2013-10-11 at 7.24.48 AMAs the UNICEF report on ICTs and C4D* notes, girls may face a number of situations that contribute to and/or are caused by their marginalization: early pregnancy, female genital cutting, early marriage, high rates of HIV/AIDS, low levels of education, lack of control over resources. ICTs alone cannot resolve these, because there is a deep and broad set of root causes. However, ICTs can be integrated systematically into the set of C4D tools and approaches that contribute to positive change.

Issues like bandwidth, censorship and electricity need to be considered when integrating ICTs into C4D work, and approaches that fit the context need to be developed. Practitioners should use tools that are in the hands of girls and their communities now, yet be aware of advances in access and new technologies, as these change rapidly.

Key points:

Interactivity is more empowering than one-way messaging:  Many of the ICT solutions being promoted today focus on sending messages out via mobile phones. However C4D approaches aim for interactivity and multi-channel, multi-directional communication, which has proven more empowering.

Content: Traditional media normally goes through a rigorous editorial process and it is possible to infuse it with a gender balance. Social media does not have the same type of filters, and it can easily be used to reinforce stereotypes about girls. This is something to watch and be aware of.

Purpose: It’s common with ICT-related approaches to start with the technology rather than starting with the goals. As one Salon participant asked “What are the results we want to see for ourselves? What are the results that girls want to see? What are the root causes of discrimination and how are we trying to address them? What does success look like for girls? For organizations? Is there a role for ICTs in helping achieve success? If so, what is it?” These questions need to be the starting point, rather than the technology.

Participation: One Salon participant mentioned a 2-year project that is working together with girls to define their needs and their vision of success. The process is one co-design, and it is aimed at understanding what girls want. Many girls expressed a feeling of isolation and desire for connection, and so the project is looking at how ICTs can help them connect. As the process developed, the diversity of needs became very clear and plans have changed dramatically based on input from a range of girls from different contexts. Implementors need to be prepared to change, adapt and respond to what girls say they want and to local realities.

****

Screen Shot 2013-11-23 at 10.41.22 PMA second study commissioned by UNICEF explores how young people use social media. The researchers encountered some challenges in terms of a strong gender approach for the study. Though a gender lens was used for analysis, there is little available data disaggregated by sex. The study does not focus on the most marginalized, because it looks at the use of social media, which normally requires a data connection or Internet access, which the most marginalized youth usually do not have.

The authors of the report found that youth most commonly used the Internet and social media for socializing and communicating with friends. Youth connected less often for schoolwork. One reason for this may be that in the countries/contexts where the research took place, there is no real integration of ICTs into the school system. It was emphasized that the  findings in the report are not comparable or nationally representative, and blanket statements such as “this means x for the whole developing world” should be avoided.

Key points:

Self-reporting biases. Boys tend to have higher levels of confidence and self-report greater ICT proficiency than girls do. This may skew results and make it seem that boys have higher skill levels.

Do girls really have less access? We often hear that girls have less access than boys. The evidence gathered for this particular report found that “yes and no.” In some places, when researchers asked “Do you have access to a mobile,” there was not a huge difference between urban and rural or between boys and girls. When they dug deeper, however, it became more complex. In the case of Zambia, access and ownership were similar for boys and girls, but fewer girls were connecting at all to the Internet as compared to boys. Understanding connectivity and use was quite complicated.

What are girls vs. boys doing online? This is an important factor when thinking about what solutions are applicable to which situation(s). Differences came up here in the study. In Argentina, girls were doing certain activities more frequently, such as chatting and looking for information, but they were not gaming. In Zambia, girls were doing some things less often than boys; for example, fewer girls than boys were looking for health information, although the number was still significant. A notable finding was that both girls and boys were accessing general health information more often than they were accessing sensitive information, such as sexual health or mental health.

What are the risks in the online world? A qualitative portion of the study in Kenya used focus groups with girls and boys, and asked about their uses of social media and experience of risk. Many out-of-school girls aged 15-17 reported that they used social media as a way to meet a potential partner to help them out of their financial situation. They reported riskier behavior, contact with older men, and relationships more often than girls who were in school. Girls in general were more likely to report unpleasant online encounters than boys, for example, request for self-exposure photos.

Hiding social media use. Most of the young people that researchers spoke with in Kenya were hiding social media use from their parents, who disapproved of it. This is an important point to note in C4D efforts that plan on using social media, and program designers will want to take parental attitudes about different media and communication channels into consideration as they design C4D programs.

****

When implementing programs, it is noteworthy how boys and girls tend to use ICT and media tools. Gender issues often manifest themselves right away. “The boys grab the cameras, the boys sit down first at the computers.” If practitioners don’t create special rules and a safe space for girls to participate, girls may be marginalized. In practical ICT and media work, it’s common for boys and girls to take on certain roles. “Some girls like to go on camera, but more often they tend to facilitate what is being done rather than star in it.” The gender gap in ICT access and use, where it exists, is a reflection of the power gaps of society in general.

In the most rural areas, even when people have access, they usually don’t have the resources and skills to use ICTs.  Very simple challenges can affect girls’ ability to participate in projects, for example, oftentimes a project will hold training at times when it’s difficult for girls to attend. Unless someone systematically goes through and applies a gender lens to a program, organizations often don’t notice the challenges girls may face in participating. It’s not enough to do gender training or measure gender once a year; gendered approaches needs to be built into program design.

Long-terms interventions are needed if the goal is to emancipate girls, help them learn better, graduate, postpone pregnancy, and get a job. This cannot be done in a year with a simple project that has only one focus, because girls are dealing with education, healthcare, and a whole series of very entrenched social issues. What’s needed is to follow a cohort of girls and to provide information and support across all these sectors over the long-term.

Key points:

Engaging boys and men: Negative reactions from men are a concern if and when girls and women start to feel more empowered or to access resources. For example, some mobile money and cash transfer programs direct funds to girls and women, and some studies have found that violence against women increases when women start to have more money and more freedom. Another study, however, of a small-scale effort that provides unconditional cash transfers to girls ages 18-19 in rural Kenya, is demonstrating just the opposite: girls have been able to say where money is spent and the gender dynamics have improved. This raises the question of whether program methodologies need to be oriented towards engaging boys and men and involving them in changing gender dynamics, and whether engaging boys and men can help avoid an increase in violence. Working with boys to become “girl champions” was cited as a way to help to bring boys into the process as advocates and role models.

Girls as producers, not just consumers. ICTs are not only tools for sending content to girls. Some programs are working to help girls produce content and create digital stories in their own languages. Sometimes these stories are used to advocate to decision makers for change in favor of girls and their agendas. Digital stories are being used as part of research processes and to support monitoring, evaluation and accountability work through ‘real-time’ data.

ICTs and social accountability. Digital tools are helping young people address accountability issues and inform local and national development processes. In some cases, youth are able to use simple, narrow bandwidth tools to keep up to date on actions of government officials or to respond to surveys to voice their priorities. Online tools can also lead to offline, face-to-face engagement. One issue, however, is that in some countries, youth are able to establish communication with national government ministers (because there is national-level capacity and infrastructure) but at local level there is very little chance or capability for engagement with elected officials, who are unprepared to respond and engage with youth or via social media. Youth therefore tend to bypass local government and communicate with national government. There is a need for capacity building at local level and decentralized policies and practices so that response capacity is strengthened.

Do ICTs marginalize girls? Some Salon participants worried that as conversations and information increasingly move to a digital environment, ICTs are magnifying the information and communication divide and further marginalizing some girls. Others felt that the fact that we are able to reach the majority of the world’s population now is very significant, and the inability to reach absolutely everyone doesn’t mean we should stop using ICTs. For this very reason – because sharing of information is increasingly digital – we should continue working to get more girls online and strengthen their confidence and abilities to use ICTs.

Many thanks to UNICEF for hosting the Salon!

(Salons operate under Chatham House Rule, thus no attribution has been given in the above summary. Sign up here if you’d like to attend Salons in the future!)

*Disclosure: I co-authored this report with Keshet Bachan.

Read Full Post »

Screen Shot 2013-10-11 at 7.24.48 AMA paper that Keshet Bachan and I authored for Unicef is now available for your reading pleasure!

Here’s a  summary of what we talk about in the paper:

Social, cultural, economic and political traditions and systems that prevent girls, especially the most marginalized, from fully achieving their rights present a formidable challenge to development organizations. The integration of new Information and Communication Technologies (ICTs) to the Communication for Development (C4D) toolbox offers an additional means for challenging unequal power relations and increasing participation of marginalized girls in social
transformation.

We examine ways that ICTs can strengthen C4D programming by:

  • enhancing girls’ connections, engagement and agency;
  • helping girls access knowledge; and
  • supporting improved governance and service delivery efforts.

We reflect and build on the views of adolescent girls from 13 developing countries who participated in a unique discussion for this paper, and we then provide recommendations to support the integration of ICTs in C4D work with marginalized adolescent girls, including:

  • Girls as active participants in program design. Practitioners should understand local context and ensure that programs use communication channels that are accessible to girls. This will often require multi-channel and multiple platform approaches that reach more marginalized girls who may not have access to or use of ICTs. Programs should be community driven, and real-time feedback from girls should be incorporated to adjust programs to their needs and preferences. Mentoring is a key component of programming with girls, and holistic programs designed together with girls tend towards being more successful.
  • Privacy and protection. Every program should conduct a thorough risk analysis of proposed approaches to ensure that girls are not placed at risk by participating, sharing and consuming information, or publicly holding others to account. Girls should also be supported to make their own informed choices about their online presence and use of ICT devices and platforms. A broader set of stakeholders should be engaged and influenced to help mitigate systemic and structural risks to girls.
  • Research and documentation. The evidence base for use of ICTs in C4D programming with marginalized adolescent girls is quite scarce. Better documentation would improve understanding of what programs are the most effective, and what the real added value of ICTs are in these efforts.
  • Capacity building. Because the integration of ICTs into C4D work is a relatively new area that lacks a consistent methodological framework, organizations should support a comprehensive training process for staff to cover areas such as program design, effective use of new ICT tools in combination with existing tools and methods, and close attention to privacy and risk mitigation.
  • Policy. Programs should use free and open source software. In addition, child protection policies, measures and guidelines should be updated to reflect changes in technology, platforms and information sharing.

The paper was first shared at the 12th Inter-Agency Roundtable on Communication for Development in November 2011. It was then reviewed and updated in August 2012, and released in August 2013 under the title “Integrating Information and Communication Technologies into Communication for Development Strategies to Support and Empower Marginalized Adolescent Girls.”

Download it here!

Read Full Post »

At Catholic Relief Services’ annual ICT4D meeting in March 2013, I worked with Jill Hannon from Rockefeller Foundation’s Evaluation Office to organize 3 sessions on the use of ICT for Monitoring and Evaluation (ICTME). The sessions covered the benefits (known and perceived) of using ICTs for M&E, the challenges and barriers organizations face when doing so, and some lessons and advice on how to integrate ICTs into the M&E process.

Our lead discussants in the three sessions included: Stella Luk (Dimagi), Guy Sharrack (CRS), Mike Matarasso (CRS), David McAfee (HNI/Datawinners), Mark Boots (Votomobile), and Teressa Trusty (USAID’s IDEA/Mobile Solutions). In addition, we drew from the experiences and expertise of some 60 people who attended our two round table sessions.

Benefits of integrating ICTs into the M&E process

Some of the potential benefits of integrating ICTs mentioned by the various discussants and participants in the sessions included:

  • More rigorous, higher quality data collection and more complete data
  • Reduction in required resources (time, human, money) to collect, aggregate and analyze data
  • Reduced complexity if data systems are simplified; thus increased productivity and efficiency
  • Combined information sources and types and integration of free form, qualitative data with quantitative data
  • Broader general feedback from a wider public via ICT tools like SMS; inclusion of new voices in the feedback process, elimination of the middleman to empower communities
  • Better cross-sections of information, information comparisons; better coordination and cross-comparing if standard, open formats are used
  • Trend-spotting with visualization tools
  • Greater data transparency and data visibility, easier data auditing
  • Real-time or near real-time feedback “up the chain” that enables quicker decision-making, adaptive management, improved allocation of limited resources based on real-time data, quicker communication of decisions/changes back to field-level staff, faster response to donors and better learning
  • Real-time feedback “down the ladder” that allows for direct citizen/beneficiary feedback, and complementing of formal M&E with other social monitoring approaches
  • Scale, greater data security and archiving, and less environmental impact
  • Better user experience for staff as well as skill enhancement and job marketability and competitiveness of staff who use the system

Barriers and challenges of integrating ICTs into M&E processes

A number of challenges and barriers were also identified, including:

  • A lack of organizational capacity to decide when to use ICTs in M&E, for what, and why, and deciding on the right ICT (if any) for the situation. Organizations may find it difficult to get beyond collecting the data to better use of data for decision-making and coordination. There is often low staff capacity, low uptake of ICT tools and resistance to change.
  • A tendency to focus on surveys and less attention to other types of M&E input, such as qualitative input. Scaling analysis of large-scale qualitative feedback is also a challenge: “How do you scale qualitative feedback to 10,000 people or more? People can give their feedback in a number of languages by voice. How do you mine that data?”
  • The temptation to offload excessive data collection to frontline staff without carefully selecting what data is actually going to be used and useful for them or for other decision-makers.
  • M&E is often tacked on at the end of a proposal design. The same is true for ICT. Both ICT and M&E need to be considered and “baked in” to a process from the very beginning.
  • ICT-based M&E systems have missed the ball on sharing data back. “Clinics in Ghana collect a lot of information that gets aggregated and moved up the chain. What doesn’t happen is sharing that information back with the clinic staff so that they can see what is happening in their own clinic and why. We need to do a better job of giving information back to people and closing the loop.” This step is also important for accountability back to communities. On the whole, we need to be less extractive.
  • Available tools are not always exactly right, and no tool seems to provide everything an organization needs, making it difficult to choose the right tool. There are too many solutions, many of which are duplicative, and often the feature sets and the usability of these tools are both poor. There are issues with sustainability and ongoing maintenance and development of M&E platforms.
  • Common definitions for data types and standards for data formatting are needed. The lack of interoperability among ICT solutions also causes challenges. As a field, we don’t do enough linking of systems together to see a bigger picture of which programs are doing what, where and who they are impacting and how.
  • Security and privacy are not adequately addressed. Many organizations or technology providers are unaware of the ethical implications of collecting data via new tools and channels. Many organizations are unclear about the ethical standards for research versus information that is offered up by different constituents or “beneficiaries” (eg., information provided by people participating in programs that use SMS or collect information through SMS-based surveys) versus monitoring and evaluation information. It is also unclear what the rules are for information collected by private companies, who this information can be shared with and what privacy laws mean for ICT-enabled M&E and other types of data collection. If there are too many barriers to collecting information, however, the amount of information collected will be reduced. A balance needs to be found. The information that telecommunications companies hold is something to think about when considering privacy and consent issues, especially in situations of higher vulnerability and risk. (UNOCHA has recently released a report that may be useful.)
  • Not enough is understood about motivation and incentive for staff or community members to participate or share data. “Where does my information go? Do I see the results? Why should I participate? Is anyone responding to my input?” In addition, the common issues of cost, access, capacity, language, literacy, cultural barriers are very much present in attempts to collect information directly from community members. Another question is that of inclusion: Does ICT-enabled data collection or surveying leave certain groups out? (See this study on intrinsic vs extrinsic motivation for feedback.)
  • Donors often push or dictate the use of ICT when it’s perhaps not the most useful for the situation. In addition there is normally not enough time during proposal process for organizations to work on buy-in and good design of an ICT-enabled M&E system. There is often a demand from the top for excessive data collection without an understanding of the effort required to collect it, and time/resource trade-offs for excessive data collection when it leads to less time spent on program implementation. “People making decisions in the capital want to add all these new questions and information and that can be a challenge… What data are valuable to collect? Who will respond to them? Who will use them as the project goes forward?”
  • There seems to be a focus on top-down, externally created solutions rather than building on local systems and strengths or supporting local organizations or small businesses to strengthen their ICTME capacities. “Can strengthening local capacity be an objective in its own right? Are donors encouraging agencies to develop vertical ICTME solutions without strengthening local systems and partners?”
  • Results-based, data-based focus can bias the countable, leave out complex development processes with more difficult to count/measure impacts.

Lessons and good practice for integrating ICTs into M&E processes

ICT is not a silver bullet – it presents its own set of challenges. But a number of good practices surfaced:

  • The use of ICTs for M&E is not just a technology issue, it’s a people and processes issue too, and it is important to manage the change carefully. It’s also important to keep an open mind that ICT4D to support M&E might not always be the best use of scarce resources – there may be more pressing priorities for a project. Getting influential people on your side to support the cause and help leverage funding and support is critical. It’s also important to communicate goals and objectives clearly, and provide incentives to make sure ICTs are successfully adopted. The trick is keeping up with technology advances to improve the system, but also keeping your eye on the ball.
  • When designing an ICTME effort, clarity of purpose and a holistic picture of the project M&E system are needed in order to review options for where ICT4D can best fit. Don’t start with the technology. Start with the M&E purpose and goals and focus on the business need, not the gadgets. Have a detailed understanding of M&E data requirements and data flows as a first step. Follow those with iterative discussions with ICT staff to specify the ICT4D solution requirements.
  • Select an important but modest project to start with and pilot in one location – get it right and work out the glitches before expanding to a second tier of pilots or expanding widely. Have a fully functional model to share for broad buy-in and collect some hard data during the pilot to convince people of adoption. The first ICT initiative will be the most important.  If it is successful, use of ICTs will likely spread throughout an organization.  If the first initiative fails, it can significantly push back the adoption of ICTs in general. For this reason, it’s important to use your best people for the first effort. Teamwork and/or new skill sets may be required to improve ICT-enabled M&E. The “ICT4D 2.0 Manifesto” talks about a tribrid set of skills needed for ICT-enabled programs.
  • Don’t underestimate the need for staff training and ongoing technical assistance to ensure a positive user experience, particularly when starting out. Agencies need to find the right balance between being able to provide support for a limited number of ICT solutions versus the need to support ongoing local innovation.  It’s also important to ask for help when needed.  The most successful M&E projects are led by competent managers who seek out resources both inside and outside their organizations.
  • Good ICT-enabled M&E comes from a partnership between program, M&E and ICT staff, technical support internal and external to the organization. Having a solid training curriculum and a good help desk are important. In addition, in-built capacity for original architecture design and to maintain and adjust the system is a good idea. A lead business owner and manager for the system need to be in place as well as global and local level pioneers and strong leadership (with budget!) to do testing and piloting. At the local level, it is important to have an energetic and savvy local M&E pioneer who has a high level of patience and understands technology.
  • At the community level, a key piece is understanding who you need to hear from for effective M&E and ensuring that ICT tools are accessible to all. It’s also critical to understand who you are ignoring or not reaching with any tool or process. Are women and children left out? What about income level? Those who are not literate?
  • Organizations should also take care that they are not replacing or obliterating existing human responsibilities for evaluation. For example, at community level in Ghana, Assembly Members have the current responsibility for representing citizen concerns. An ICT-enabled feedback loop might undermine this responsibility if it seeks direct-from-citizen evaluation input.  The issue of trust and the human-human link also need consideration. ICT cannot and should not be a replacement for everything. New ICT tools can increase the number of people and factors evaluated; not just increase efficiency of existing evaluations.
  • Along the same lines, it’s important not to duplicate existing information systems, create parallel systems or fragment the government’s own systems. Organizations should be strengthening local government systems and working with government to use the information to inform policy and help with decision-making and implementation of programs.
  • implementors need to think about the direction of information flow. “Is it valuable to share results “upward” and “downward”? It is possible to integrate local decision-making into a system.” Systems can be created that allow for immediate local-level decision-making based on survey input. Key survey questions can be linked to indicators that allow for immediate discussion and solutions to improve service provision.
  • Also, the potential political and social implications of greater openness in information flows needs to be considered. Will local, regional and national government embrace the openness and transparency that ICTs offer? Are donors and NGOs potentially putting people at risk?
  • For best results, pick a feasible and limited number of quality indicators and think through how frontline workers will be motivated to collect the data. Excessive data collection will interfere with or impede service delivery. Make sure managers are capable of handling and analyzing data that comes in and reacting to it, or there is no point in collecting it. It’s important to not only think about what data you want, but how this data will be used. Real-time data collected needs to be actionable. Be sure that those submitting data understand what data they have submitted and can verify its accuracy. Mobile data collection needs to be integrated into real processes and feedback loops. People will only submit information or reports if they see that someone cares about those reports and does something about them.
  • Collecting data through mobile technology may change the behavior being monitored or tracked. One participant commented that when his organization implemented an ICT-based system to track staff performance, people started doing unnecessary activities so that they could tick off the system boxes rather than doing what they knew should be done for better program impact.
  • At the practical level, tips include having robust options for connectivity and power solutions, testing the technology in the field with a real situation, securing reduced costs with vendors for bulk purchasing and master agreements, using standard vendor tools instead of custom building. It’s good to keep the system simple, efficient and effective as possible and to avoid redundancy or the addition of features things that don’t truly offer more functionality.

Thanks to all our participants and lead discussants at the sessions!

Useful information and guides on ICTME:

Mobile-based technology for monitoring and evaluation: A reference guide for project managers, M&E specialists, researchers, donors

3 Reports on mobile data collection

Other posts on ICTs for M&E:

12 tips on using ICTs for social monitoring and accountability

11 points on strengthening local capacity to use new ICTs for M&E

10 tips on using new ICTs for qualitative M&E

Using participatory video for M&E

ICTs and M&E at the South Asia Evaluators’ Conclave

Read Full Post »

« Newer Posts - Older Posts »