Archive for the ‘wait… what?’ Category

At our November 18th Technology Salon, we discussed how different organizations are developing their ICT for development (ICT4D) strategies. We shared learning on strategy development and buy-in, talked about whether organizations should create special teams or labs for ICT- and innovation-related work or mainstream the ICT4D function, and thought about how organizations can define and find the skill sets needed for taking their ICT-enabled work forward. Population Council’s Stan Mierzwa, Oxfam America’s Neal McCarthy, and Cycle Technologies’ Leslie Heyer joined as lead discussants, and we heard from Salon participants about their experiences too.

Participating organizations were at various stages of ICT4D work, but most had experienced similar challenges and frustrations with taking their work forward. Even organizations that had created ICT4D strategies a couple of years ago said that implementation was slow.

Some of the key elements mentioned by our first discussant as important for managing and strategically moving ICT forward in an organization included:

  • being more informed about where different offices and staff were using ICTs for programmatic work,
  • establishing a standard set of technology tools for organizational use,
  • improved knowledge management about ICTs,
  • publishing on how ICTs were being used in programs (to help with credibility),
  • engaging with different teams and leadership to secure support and resources
  • working more closely with human resources teams who often do not understand ICT4D-related job descriptions and the profile needed.

Our second discussant said that his organization developed an ICT4D strategy in order to secure resources and greater support for moving ICT4D forward. It was also starting to be unwieldy to manage all of the different ideas and tools being used across the organization, and it seemed that greater harmonization would allow for improved IT support for more established tools as well as establishment of other ways to support new innovations.

In this case, the organization looked at ICTs as two categories: technology for development workers and technology for development outcomes. They used Gartner’s ‘pace layered’ model (which characterizes systems of innovation, systems of differentiation, and systems of record) as a way of analyzing the support roles of different departments.

One of the initial actions taken by this organization was establishing a small tech incubation fund that different offices could apply for in order to try something new with ICTs in their programs and campaigns. Another action was to take 10 staff to the Catholic Relief Services (CRS) ICT4D conference to learn more about ICT4D and to see what their peers from similar organizations were doing. In return for attending the conference, staff were required to submit a proposal for the tech incubation fund.

For the development of the strategy document and action plan, the ICT4D strategy team worked with a wider group of staff to develop a list of current ICT-enabled initiatives and a visual heat map of actions and activities across the organization. This formed the basis for discussions on where lots of ICT4D activities were happening and where there was nothing going on with ICTs. The team then discussed what the organization should do strategically to support and potentially consolidate existing activities and what should be done about areas where there were few ICT-related activities – should those areas be left alone or was there a reason to look at them to see if ICT should be incorporated?

Having done that, the organization adapted Nethope’s Organizational Guide to ICT4D to fit its own structure and culture, and used it as a framework for ICT4D strategy discussions with key staff from different teams. The Nethope guide suggests five key functions for strategic, organization-wide ICT4D: lead organizational change, drive knowledge exchange, build a portfolio, manage processes, and develop an advisory service (see below). The aforementioned activities were also clustered according to which of these 5 areas they fell into.

Screen Shot 2015-11-24 at 8.53.12 AM

(Table of contents from Nethope’s Guide.)

The organization felt it was also important to change the image of the IT team. ‘We had to show that we were not going to tie people up with formal committees and approvals if they wanted to try something new and innovative. Being more approachable is necessary or staff will bypass the IT team and go to consultants, and then we open ourselves up to data privacy risks and we also lose institutional knowledge.’

Salon participants agreed that it was important to know how to “sell” an ICT4D-related idea to frontline staff, management and leadership. Some ways to do this include demonstrating the value-add of ICTs in terms of longer-term cost and time efficiencies, showing the benefit of real-time data for decision-making, and demonstrating what peer organizations are doing. Organizations often also need someone at the top who is pushing for change and modernization.

Our third discussant said that her company has been shifting from a commercial product developer to a full-fledged technology company. She outlined the need for strategic thinking along that journey. Initially, the company outsourced activities such as research and data collection. With time, it started to pull key functions in house since systems maintenance and technology has become a core part of the business.

“As a small company, we can be flexible and change easily,” she said. ‘ICT is embedded into our culture and everyone thinks about it.’ One challenge that many ICT4D initiatives face – whether they are happening in a non-profit or a for-profit — is sustainability. ‘People are often fine with paying for a physical product, but when it comes to the web, they are accustomed to getting everything for free, which makes long-term sustainability difficult.’

In order to continuously evolve their strategies, organizations need to have time and space to constantly step back and think about their underlying values and where they see themselves in 5 or 10 years. A more pro-active relationship with donors is also important. Although Salon participants felt that the ICT4D Principles and related processes were promising, they also felt that donors do not have a clear idea of what they are looking for, what exists already, what needs to be created, and what evidence base exists for different tools or kinds of ICT4D. One Salon participant felt that ‘donor agencies don’t know what kinds of tech are effective, so it’s up to you as an implementer to bring the evidence to the table. It’s critical to have the ITC4D support staff at the table with you, because if not these more detailed conversations about the tech don’t happen with donors and you’ll find all kinds of duplication of efforts.’

Another challenge with thinking about ICT4D in a strategic way is that donors normally don’t want to fund capacity building, said another Salon participant. They prefer to fund concrete projects or innovation challenges rather than supporting organizations to create an environment that gives rise to innovation. In addition, funding beyond the program cycle is a big challenge. ‘We need to be thinking about enterprise systems, layered on systems, national systems,’ said one person. ‘And systems really struggle here to scale and grow if you can’t claim ownership for the whole.’

Salon participants highlighted hiring and human resources departments as a big barrier when it comes to ICT4D. It is often not clear what kinds of skills are needed to implement ICT4D programs, and human resources teams often screen for the wrong skill sets because they do not understand the nature of ICT4D. ‘I always make them give me all the CVs and screen them myself,’ said one person. ‘If not, some of the best people will not make it to the short list.’ Engaging with human resources and sharing the ICT4D strategy is one way to help with better hiring and matching of job needs with skill sets that are out there and potentially difficult to find.

In conclusion, whether the ICT4D strategy is to mainstream, to isolate and create a ‘lab,’ or to combine approaches, it seems that most organizations are struggling a bit to develop and/or implement ICT4D strategies due to the multiple pain points of slow organizational change and the need for more capacity and resources. Some are making headway, however, and developing clearer thinking and action plans that are paying off in the short term, and that may set the organizations up for eventual ICT4D success.

Thanks to Population Council for hosting this Salon! If you’d like to join discussions like this one, sign up at Technology Salon.

Salons are held under Chatham House Rule. No attribution has been made in this post.

Read Full Post »

Traditional development evaluation has been characterized as ‘backward looking’ rather than forward looking and too focused on proving over improving. Some believe applying an ‘agile’ approach in development would be more useful — the assumption being that if you design a program properly and iterate rapidly and constantly based on user feedback and data analytics, you are more likely achieve your goal or outcome without requiring expensive evaluations. The idea is that big data could eventually allow development agencies to collect enough passive data about program participants that there would no longer be a need to actively survey people or conduct a final evaluation, because there would be obvious patterns that would allow implementers to understand behaviors and improve programs along the way.

The above factors have made some evaluators and data scientists question whether big data and real-time availability of multiple big data sets, along with the technology that enables their collection and analysis, will make evaluation as we know it obsolete. Others have argued that it’s not the end of evaluation, but rather we will see a blending of real-time monitoring, predictive modeling, and impact evaluation, depending on the situation. Big questions remain, however, about the feasibility of big data in some contexts. For example, are big data approaches useful when it comes to people who are not producing very much digital data? How will the biases in big data be addressed to ensure that the poorest, least connected, and/or most marginalized are represented?

The Technology Salon on Big Data and Evaluation hosted during November’s  American Evaluation Association Conference in Chicago opened these questions up for consideration by a roomful of evaluators and a few data scientists. We discussed the potential role of new kinds and quantities of data. We asked how to incorporate static and dynamic big data sources into development evaluation. We shared ideas on what tools, skills, and partnerships we might require if we aim to incorporate big data into evaluation practice. This rich and well-informed conversation was catalyzed by our lead discussants: Andrew Means, Associate Director of the Center for Data Science & Public Policy at the University of Chicago and Founder of Data Analysts for Social Good and The Impact Lab; Michael Bamberger, Independent Evaluator and co-author of Real World Evaluation; and Veronica Olazabal from The Rockefeller Foundation. The Salon was supported by ITAD via a Rockefeller Foundation grant.

What do we mean by ‘big data’?

The first task was to come up with a general working definition of what was understood by ‘big data.’ Very few of the organizations present at the Salon were actually using ‘big data’ and definitions varied. Some talked about ‘big data sets’ as those that could not be collected or analyzed by a human on a standard computer. Others mentioned that big data could include ‘static’ data sets (like government census data – if digitized — or cellphone record data) and ‘dynamic’ data sets that are being constantly generated in real time (such as streaming data input from sensors or ‘cookies’ and ‘crumbs’ generated through use of the Internet and social media). Others considered big data to be real time, socially-created and socially-driven data that could be harvested without having to purposely collect it or budget for its collection. ‘It’s data that has a life of its own. Data that just exists out there.’ Yet others felt that for something to be ‘big data’ multiple big data sets needed to be involved, for example, genetic molecular data crossed with clinical trial data and other large data sets, regardless of static or dynamic nature. Big data, most agreed, is data that doesn’t easily fit on a laptop and that requires a specialized skill set that most social scientists don’t have. ‘What is big data? It’s hard to define exactly, but I know it when I see it,’ concluded one discussant.

Why is big data a ‘thing’?

As one discussant outlined, recent changes in technology have given rise to big data. Data collection, data storage and analytical power are becoming cheaper and cheaper. ‘We live digitally now and we produce data all the time. A UPS truck has anywhere from 50-75 sensors on it to do everything from optimize routes to indicate how often it visits a mechanic,’ he said. ‘The analytic and computational power in my iPhone is greater than what the space shuttle had.’ In addition, we have ‘seamless data collection’ in the case of Internet-enabled products and services, meaning that a person creates data as they access products or services, and this can then be monetized, which is how companies like Google make their money. ‘There is not someone sitting at Google going — OK, Joe just searched for the nearest pizza place, let me enter that data into the system — Joe is creating the data about his search while he is searching, and this data is a constant stream.’

What does big data mean for development evaluation?

Evaluators are normally tasked with making a judgment about the merit of something, usually for accountability, learning and/or to improve service delivery, and usually looking back at what has already happened. In the wider sense, the learning from evaluation contributes to program theory, needs assessment, and many other parts of the program cycle.

This approach differs in some key ways from big data work, because most of the new analytical methods used by data scientists are good at prediction but not very good at understanding causality, which is what social scientists (and evaluators) are most often interested in. ‘We don’t just look at giant data sets and find random correlations,’ however, explained one discussant. ‘That’s not practical at all. Rather, we start with a hypothesis and make a mental model of how different things might be working together. We create regression models and see which performs better. This helps us to know if we are building the right hypothesis. And then we chisel away at that hypothesis.’

Some challenges come up when we think about big data for development evaluation because the social sector lacks the resources of the private sector. In addition, data collection in the world of international development is not often seamless because ‘we care about people who do not live in the digital world,’ as one person put it. Populations we work with often do not leave a digital trail. Moreover, we only have complete data about the entire population in some cases (for example, when it comes to education in the US), meaning that development evaluators need to figure out how to deal with bias and sampling.

Satellite imagery can bring in some data that was unavailable in the past, and this is useful for climate and environmental work, but we still do not have a lot of big data for other types of programming, one person said. What’s more, wholly machine-based learning, and the kind of ‘deep learning’ made possible by today’s computational power is currently not very useful for development evaluation.

Evaluators often develop counterfactuals so that they can determine what would have happened without an intervention. They may use randomized controlled trials (RCTs), differentiation models, statistics and economics research approaches to do this. One area where data science may provide some support is in helping to answer questions about counterfactuals.

More access to big data (and open data) could also mean that development and humanitarian organizations stop duplicating data collection functions. Perhaps most interestingly, big data’s predictive capabilities could in the future be used in the planning phase to inform the kinds of programs that agencies run, where they should be run, and who should be let into them to achieve the greatest impact, said one discussant. Computer scientists and social scientists need to break down language barriers and come together more often so they can better learn from one another and determine where their approaches can overlap and be mutually supportive.

Are we all going to be using big data?

Not everyone needs to use big data. Not everyone has the capacity to use it, and it doesn’t exist for offline populations, so we need to be careful that we are not forcing it where it’s not the best approach. As one discussant emphasized, big data is not magic, and it’s not universally applicable. It’s good for some questions and not others, and it should be considered as another tool in the toolbox rather than the only tool. Big data can provide clues to what needs further examination using other methods, and thus most often it should be part of a mixed methods approach. Some participants felt that the discussion about big data was similar to the one 20 years ago on electronic medical records or to the debate in the evaluation community about quantitative versus qualitative methods.

What about groups of people who are digitally invisible?

There are serious limitations when it comes to the data we have access to in the poorest communities, where there are no tablets and fewer cellphones. We also need to be aware of ‘micro-exclusion’ (who within a community or household is left out of the digital revolution?) and intersectionality (how do different factors of exclusion combine to limit certain people’s digital access?) and consider how these affect the generation and interpretation of big data. There is also a question about the intensity of the digital footprint: How much data and at what frequency is it required for big data to be useful?

Some Salon participants felt that over time, everyone would have a digital presence and/or data trail, but others were skeptical. Some data scientists are experimenting with calibrating small amounts of data and comparing them to human-collected data in an attempt to make big data less biased, a discussant explained. Another person said that by digitizing and validating government data on thousands (in the case of India, millions) of villages, big data sets could be created for those that are not using mobiles or data.

Another person pointed out that generating digital data is a process that involves much more than simple access to technology. ‘Joining the digital discussion’ also requires access to networks, local language content, and all kinds of other precursors, she said. We also need to be very aware that these kinds of data collection processes impact on people’s participation and input into data collection and analysis. ‘There’s a difference between a collective evaluation activity where people are sitting around together discussing things and someone sitting in an office far from the community getting sound bites from a large source of data.’

Where is big data most applicable in evaluation?

One discussant laid out areas where big data would likely be the most applicable to development evaluation:

Screen Shot 2015-11-23 at 9.32.07 AM

It would appear that big data has huge potential in the evaluation of complex programs, he continued. ‘It’s fairly widely accepted that conventional designs don’t work well with multiple causality, multiple actors, multiple contextual variables, etc. People chug on valiantly, but it’s expected that you may get very misleading results. This is an interesting area because there are almost no evaluation designs for complexity, and big data might be a possibility here.’

In what scenarios might we use big data for development evaluation?

This discussant suggested that big data might be considered useful for evaluation in three areas:

  1. Supporting conventional evaluation design by adding new big data generated variables. For example, one could add transaction data from ATMs to conventional survey generated poverty indicators
  2. Increasing the power of a conventional evaluation design by using big data to strengthen the sample selection methodology. For example, satellite images were combined with data collected on the ground and propensity score matching was used to strengthen comparison group selection for an evaluation of the effects of interventions on protecting forest cover in Mexico.
  3. Replacing a conventional design with a big data analytics design by replacing regression based models with systems analysis. For example, one could use systems analysis to compare the effectiveness of 30 ongoing interventions that may reduce stunting in a sample of villages. Real-time observations could generate a time-series that could help to estimate the effectiveness of each intervention in different contexts.

It is important to remember construct validity too. ‘If big data is available, but it’s not quite answering the question that you want to ask, it might be easy to decide to do something with it, to run some correlations, and to think that maybe something will come out. But we should avoid this temptation,’ he cautioned. ‘We need to remember and respect construct validity and focus on measuring what we think we are measuring and what we want to measure, not get distracted by what a data set might offer us.’

What about bias in data sets?

We also need to be very aware that big data carries with it certain biases that need to be accounted for, commented several participants; notably, when working with low connectivity populations and geographies or when using data from social media sites that cater to a particular segment of the population. One discussant shared an example where Twitter was used to identify patterns in food poisoning, and suddenly the upscale, hipster restaurants in the city seemed to be the problem. Obviously these restaurants were not the sole source of the food poisoning, but rather there was a particular kind of person that tended to use Twitter.

‘People are often unclear about what’s magical and what’s really possible when it comes to big data. We want it to tell us impossible things and it can’t. We really need to engage human minds in this process; it’s not a question of everything being automated. We need to use our capacity for critical thinking and ask: Who’s creating the data? How’s it created? Where’s it coming from? Who might be left out? What could go wrong?’ emphasized one discussant. ‘Some of this information can come from the metadata, but that’s not always enough to make certain big data is a reliable source.’ Bias may also be introduced through the viewpoints and unconscious positions, values and frameworks of the data scientists themselves as they are developing algorithms and looking for/finding patterns in data.

What about the ethical and privacy implications?

Big Data has a great deal of ethical and privacy implications. Issues of consent and potential risk are critical considerations, especially when working with populations that are newly online and/or who may not have a good understanding of data privacy and how their data may be used by third parties who are collecting and/or selling it. However, one participant felt that a protectionist mentality is misguided. ‘We are pushing back and saying that social media and data tracking are bad. Instead, we should realize that having a digital life and being counted in the world is a right and it’s going to be inevitable in the future. We should be working with the people we serve to better understand digital privacy and help them to be more savvy digital citizens.’ It’s also imperative that aid and development agencies abandon our slow and antiquated data collection systems, she said, and to use the new digital tools that are available to us.

How can we be more responsible with the data we gather and use?

Development and humanitarian agencies do need be more responsible with data policies and practices, however. Big data approaches may contribute to negative data extraction tendencies if we mine data and deliver it to decision-makers far away from the source. It will be critical for evaluators and big data practitioners to find ways to engage people ‘on the ground’ and involve more communities in interpreting and querying their own big data. (For more on responsible data use, see the Responsible Development Data Book. Oxfam also has a responsible data policy that could serve as a reference. The author of this blog is working on a policy and practice guide for protecting girls digital safety, security and privacy as well.)

Who should be paying for big data sets to be made available?

One participant asked about costs and who should bear the expense of creating big data sets and/or opening them up to evaluators and/or data scientists. Others asked for examples of the private sector providing data to the social sector. This highlighted additional ethical and privacy issues. One participant gave an example from the healthcare space where there is lots of experience in accessing big data sets generated by government and the private sector. In this case, public and private data sets needed to be combined. There were strict requirements around anonymization and the effort ended up being very expensive, which made it difficult to build a business case for the work.

This can be a problem for the development sector, because it is difficult to generate resources for resolving social problems; there is normally only investment if there is some kind of commercial gain to be had. Some organizations are now hiring ‘data philanthropist’ positions that help to negotiate these kinds of data relationships with the private sector. (Global Pulse has developed a set of big data privacy principles to guide these cases.)

So, is big data going to replace evaluation or not?

In conclusion, big data will not eliminate the need for evaluation. Rather, it’s likely that it will be integrated as another source of information for strengthening conventional evaluation design. ‘Big Data and the underlying methods of data science are opening up new opportunities to answer old questions in new ways, and ask new kinds of questions. But that doesn’t mean that we should turn to big data and its methods for everything,’ said one discussant. ‘We need to get past a blind faith in big data and get more practical about what it is, how to use it, and where it adds value to evaluation processes,’ said another.

Thanks again to all who participated in the discussion! If you’d like to join (or read about) conversations like this one, visit Technology Salon. Salons run under Chatham House Rule, so no attribution has been made in this summary post.

Read Full Post »

I had the privilege (no pun intended) of participating in the Art-a-Hack program via ThoughtWorks this past couple of months. Art-a-Hack is a creative space for artists and hackers to get together for 4 Mondays in June and work together on projects that involve art, tech and hacking. There’s no funding involved, just encouragement, support, and a physical place to help you carve out some time out for discovery and exploration.

I was paired up by the organizers with two others (Dmytri and Juan), and we embarked on a project. I had earlier submitted an idea of the core issues that I wanted to explore, and we mind-melded really well to come up with a plan to create something around them.

Here is our press release with links to the final product – WhiteSave.me. You can read our Artist Statement here and follow us on Twitter @whitesave.me. Feedback welcome, and please share if you think it’s worth sharing. Needless to say full responsibility for the project falls with the team, and it does not represent the views of any past, present or future employers or colleagues.


Announcing WhiteSave.me

WhiteSave.me is a revolutionary new platform that enables White Saviors to deliver privilege to non-Whites whenever and wherever they need it with the simple tap of a finger.

Today’s White guy is increasingly told “check your privilege.” He often asks himself “What am I supposed to do about my privilege? It’s not my fault I was born white! And really, I’m not a bad person!”

Until now, there has been no simple way for a White guy to be proactive in addressing the issue of his privilege. He’s been told that he benefits from biased institutions and that his privilege is related to historically entrenched power structures. He’s told to be an ally but advised to take a back seat and follow the lead from people of color. Unfortunately this is all complex and time consuming, and addressing privilege in this way is hard work.

We need to address the issue of White privilege now however – we can’t wait. Changing attitudes, institutions, policies and structures takes too damn long! What’s more, we can’t expect White men or our current systems to go through deep changes in order to address privilege and inequality at the roots. What we can do is leapfrog over what would normally require decades of grassroots social organizing, education, policy work, and behavior change and put the solution to White privilege directly into White men’s hands so that everyone can get back to enjoying the American dream.

Screen Shot 2015-07-24 at 5.31.10 PM

WhiteSave.me – an innovative solution that enables White men to quickly and easily deliver privilege to the underprivileged, requiring only a few minutes of downtime, at their discretion and convenience.

Though not everyone realizes it, White privilege affects a large number of White people, regardless of their age or political persuasion. White liberals generally agree that they are privileged, but most are simply tired of hearing about it and having to deal with it. Conservative White men believe their privilege is all earned, but most also consider it possible to teach people of color about deep-seated American values and traditions and the notion of personal responsibility. All told, what most White people want is a simple, direct way to address their privilege once and for all. Our research has confirmed that most White people would be willing to spend a few minutes every now and then sharing their privilege, as long as it does not require too much effort.

WhiteSave.me is a revolutionary and innovative way of addressing this issue. (Read Our Story here to learn more about our discovery moments!) We’ve designed a simple web and mobile platform that enables White men to quickly and easily deliver a little bit of their excess privilege to non-Whites, all through a simple and streamlined digital interface. Liberal Whites can assuage guilt and concern about their own privilege with the tap of a finger. Conservatives can feel satisfied that they have passed along good values to non-Whites. Libertarians can prove through direct digital action that tech can resolve complex issues without government intervention and via the free market. And non-White people of any economic status, all over the world, will benefit from immediate access to White privilege directly through their devices. Everyone wins – with no messy disruption of the status quo!

How it Works

Visit our “how it works” page for more information, or simply “try it now” and your first privilege delivery session is on us! Our patented Facial Color Recognition Algorithm (™) will determine whether you qualify as a White Savior, based on your skin color. (Alternatively it will classify you as a non-White ‘Savee’). Once we determine your Whiteness, you’ll be automatically connected via live video with a Savee who is lacking in White privilege so that you can share some of your good sense and privileged counsel with him or her, or periodically alleviate your guilt by offering advice and a one-off session of helping someone who is less privileged.

Our smart business model guarantees WhiteSave.me will be around for as long as it’s needed, and that we can continue innovating with technology to iterate new solutions as technology advances. WhiteSave.me is free for White Saviors to deliver privilege, and non-Whites can choose from our Third World Freemium Model (free), our Basic Model ($9/month), or our Premium Model ($29/month). To generate additional revenue, our scientific analysis of non-White user data will enable us to place targeted advertisements that allow investors and partners to extract value from the Base of the Pyramid. Non-Profit partners are encouraged to engage WhiteSave.me as their tech partner for funding proposals, thereby appearing innovative and guaranteeing successful grant revenue.

See our FAQs for additional information and check out our Success Stories for more on how WhiteSave.me, in just its first few months, has helped thousands to deliver privilege all over the world.

Try It Now and you’ll be immediately on your way to delivering privilege through our quick and easy digital solution!

Contact help@whitesave.me for more information. And please help us spread the word. Addressing the issue of White privilege has never been so easy!


Read Full Post »

I love Dr. Seuss. His books are creative and zany. He made great social commentary. “If I Ran the Zoo” is a story about innovation and re-invention*. The hero, Gerald McGrew, is a young a boy who re-imagines the zoo. In his vision for the new zoo, he travels the world to find cool creatures that no one has ever seen. He brings them back to showcase in his “new zoo McGrew zoo,” which is dynamic, flashy and exciting.

McGrew’s new zoo looks a lot like today’s world of development sector innovation and “innovation for social good.” Great ideas and discoveries; fresh things to look at, play with and marvel at; but also quite laden with an adolescent boy’s special brand of ego and hubris.

See, most of our institutions have been basically like this for a while:

Screen Shot 2015-06-12 at 2.49.53 PM

But over the past decade, we’ve been hearing quite a lot of this:

Screen Shot 2015-06-12 at 2.52.14 PM

People inside and outside of the development and social sectors are innovating really hard to come up with new and cool things. Silicon Valley is putting in its two cents and inventing “life-changing solutions.” People are traveling all around and looking for “local” innovation, too. Some donors are even are supporting what they like to call “reverse innovation.” It feels a bit like the days of colonization are rolling on and on.Screen Shot 2015-06-12 at 1.49.18 PM

We see people with resources exploring and looking for opportunities, amazing ideas, and places to invest in or extract out value (BOP anyone?). These new ideas and innovations are captured and showcased for donors, investors, and global development peers.

Screen Shot 2015-06-12 at 1.51.59 PM

The most innovative are applauded and given more resources. Those who “win” at innovation are congratulated on Ted stages, like McGrew is for his cool new flavor of exotic creatures.

Screen Shot 2015-06-12 at 1.53.21 PM

But it’s fairly safe to say that one of the biggest problems in the world today is inequality. Many believe it’s the development model (in the small and the big sense) itself that’s the problem. Yet most of this “innovation for social good” is being stimulated by and developed within the capitalist, colonial, patriarchal models and structures that entrench inequality in the first place.

If I ran the zoo, I’d take innovation in a different direction. I’d try to figure out how to dismantle the zoo.


Fun fact: Many people credit Dr. Seuss with coining the term ‘nerd’ in this book.

(Screenshots from: https://www.youtube.com/watch?t=20&v=BLQpqkbsrr0 and https://books.google.com/books?id=fdX3xUSbriIC&pg=PT57&source=gbs_selected_pages&cad=3#v=onepage&q&f=false. Book by Dr. Seuss from 1950)

Read Full Post »

The private sector has been using dashboards for quite some time, but international development organizations face challenges when it comes to identifying the right data dashboards and accompanying systems for decision-making.

Our May 29th, 2015, Technology Salon (sponsored by The Rockefeller Foundation) explored data dashboards and data visualization for improved decision making with lead discussants John DeRiggi, Senior Data Architect, DAI; Shawna Hoffman, Associate Manager, Evaluation and Learning at The MasterCard Foundation; Stephanie Evergreen, Evergreen Data.

In short, we learned at the Salon that most organizations are struggling with the data dashboard process. There are a number of reasons that dashboards fail. They may never get off the ground, they may not deliver what was promised, they may deliver but no one uses them, or they may deliver but the data is poor and bad decisions are made. Using data for better decision-making is an ongoing process – not a task or product to complete and then relegate to automation. Just getting a dashboard up and running doesn’t guarantee that it’s a success – it’s critical to look deeper to see if the data and its visualization have actually improved decisions and how. Like with any ICT tool, user centered design and ongoing iteration are key. Successful dashboards are organized, useful, include targets, and have trends and predictions. Organizational culture and change management are critical in the process.

Points discussed in detail*:

1) Ask whether you actually need a dashboard

The first question to ask is whether a dashboard is needed or possible. One discussant, who specializes in data visualization, noted that she’s often brought in because someone wants to do data visualization, and she then needs to work backwards with the organization through a number of other preparatory steps before getting to the part on data visualization. It’s critical to have data dashboard discussions with different parts of the organization in order to understand real needs and expectations. Often people will say they need a dashboard because they want to make better decisions, noted another lead discussant. “But what kind of decisions, and what information is needed to make those decisions? Where does that information come from? Who will get it?”

2) Define the audience and type of dashboard

People often think that they can create one dashboard that will fulfill everyone’s needs. As one discussant put it, they will say the audience for the dashboard is “everyone – all decision makers at all levels!” In reality most organizations will need several dashboards for different levels of decision-making. It’s important to know who will own it, use it, keep it up, and collect the data. Will it be internal or externally facing? Discussing all of this is a key part of the process of thinking through the dashboard. As one discussant outlined, dashboards can be strategic, analytical or operational. But it’s difficult for them to be all three at once. So organizations need to come to a clear understanding of their data and decision-making needs. What information, if available, would help different teams at different levels with their decision making? One dashboard can’t be everything to everyone. Creating a charter that outlines what the dashboard project is and what it aims to do is a way to help avoid mission creep, said one discussant.

3) Work with users to develop your dashboard

To start off the process, it’s important to clearly identify the audience and find out what they need – don’t assume you know, recommended one discussant. But also, as a Salon participant pointed out, don’t assume that they know either. Have a conversation where their and your expertise comes together. “The higher up you go, the less people may understand about data. One idea is to just take the ‘data’ out of the conversation. Ask decision-makers what questions they are trying to answer, what problems they are trying to solve. Then find out how to collect and visualize the data that helps them answer their questions,” suggested another participant. Create ownership and accountability at all levels – with users, with staff who will input the data, with project managers, with grantees – you need cooperation from all levels noted others. Clear buy-in will also help with data quality. If people see the results of their data coming out in a data visualization, they may be more inclined to provide quality data. One way to involve users is to gather different teams to talk about their data and to create ‘entity relationship models’ together. “People can get into the weeds, and then you can build a vocabulary for the organization. Then you can use that model to build the system and create commonality across it,” said one discussant. Another idea is to create paper prototypes of dashboards with users so that they can envision them better.

4) Dashboards help people engage with the data they’ve collected

A dashboard is a window into your data, said one participant. In some cases, seeing their data visualized can help staff to see that they have been providing poor quality data. “People didn’t realize how bad their data was until they saw their dashboard,” said one discussant. Another noted that people may disagree with what the data tells them in the dashboard and feel motivated to provide better data. On the other hand, they may realize that their data was actually good, and instead they need to improve ineffective programs. A danger is that putting a dashboard on top of bad data shines a light on the data, said one participant, and this might create an incentive for people to manipulate their data.

5) Don’t be over-ambitious

Align the dashboard with indicators that link to strategic goals and directions and stay focused, recommended one discussant. There is often a temptation to over-complicate with tons of data and visuals. But extraneous data leads to misinterpretation or distraction. Dashboards should make complex data available in an accessible way to users, she said. You can always make more visuals if needed, but you want a concise story told in the data and visuals that you’re depicting. Determine what is useful, productive and credible and leave out what is exciting but extraneous. “Don’t try to have 30 indicators.”

6) Be clear about your data categories and indicators

Rolling up data from a large number of different programs into a dashboard is a huge challenge, especially if different sites or programs are using different data models. For example, if one program is describing an activity as a ‘workshop’ and the other uses ‘training session,’ said one discussant, you have a problem. A Salon participant explained that her organization started with shallow but important common denominators across programs. Over time they aim to go deeper to begin looking at outcomes and impact.

7) Think through how you’ll sustain the dashboard and related system(s)

One discussant said that her organization established three different teams to work on the dashboard process: a) Metrics – Where do we have credible representative data? Where do we have indicators but we don’t have data? b) Plumbing: Where are the data sources? How do they feed into each other? Who is responsible, and can this be aggregated up? And c) Visualization: What visual would help different decision makers make their decisions? Depending on where the organization is in its stage of readiness and its existing staff capacities, different combinations of skill sets may be required to supplement existing ones. Data experts can help teams understand what is possible, yet program or management teams and other dashboard users also need to be involved so that they can identify the questions they are trying to answer with the data and the dashboard.

8) Don’t underestimate the time/resources needed for a functional dashboard

People may not realize that you can’t make a dashboard without data to support it, noted one participant. “It’s like a power point presentation… a power point doesn’t just appear out of nowhere. It’s a result of conversations, research, data, design and more. But for some reason, people think a dashboard will just magically create itself out of thin air.” People also seem to think you can create and launch a dashboard and then put it on autopilot, but that is not the case. The dashboard will need constant changes and iteration, and there will be continual work to keep it up. The questions being asked will also likely change over time and so the dashboard may need to shift to take this into consideration. Time will be required to get buy-in for the dashboard and its use. One Salon participant said that in her former organization, they met quarterly to present, use and discuss the dashboard, and it took about 2 years in order for it to become useful and for people to become invested in it. It’s very important, said one participant, to ensure that management knows that the dashboard is not a static thing – it will need ongoing attention and management.

9) Be selective when it comes to the technology

People tend to think that dashboards are just visual, said a Salon participant. They think they are really cool, business solution platforms. Often senior leadership has seen been pitched something really expensive and complicated, with all kinds of bells and whistles, and they may think that is what they need. It’s important to know where your organization is in terms of capacity before determining which technology would be the best fit, however, noted one discussant. She counseled organizations to use whatever they have on hand rather than bringing in new software that takes people 6 months to learn how to use. Simple excel-based dashboards might be the best place to start, she said.

10) Legacy systems can be combined with new data viz capabilities

One discussant shared how his company’s information system, which was set up over 15 years ago, did not allow for the creation of APIs. This meant that the team could not build derivative software products from their massive existing database. It is too expensive to replace the entire system, and building modules to replace some of it would lead to fragmenting the user experience. So the team built a thin web service layer on top of the existing system. This exposed the data to friendly web formats from which developers could build interactive products.

11) Be realistic about “real time” and “data quality”

One question that came up was around the the level of evidence needed to make good decisions. Having perfect data served up into a perfect visualization is utopian, said one Salon participant. The idea is that we could have ‘real time’ data to inform our decisions, she explained, yet it’s hard to quality check data so quickly. “So at what level can we say we’ll make decisions based on a level of certainty – is it when we feel 80% of the data is good quality? Do we need to lower that to 60% so that we have timely data? Is that too low?” Another question was around the kinds of decisions that require ‘real time’ data versus those that could be made based on data that is 3 to 6 months old. Salon participants said this will depend on the kind of program and the type of decision. The sector in which one is working may also determine the level of comfort with real time and with data quality – for example, the humanitarian sector may need more timely data and accept a lower level of verification whereas the development sector may be the opposite.

Another point was that dashboards should include error bars and available metadata, as well as in some cases a link to raw data for those who want to dig into the data and understand what is behind the dashboard. Sometimes the dashboard process will highlight that there is simply not much quality data available for some programs in some countries. This can be an opportunity to work with staff on the ground to strengthen capacity to collect it.

12) Relax

As one discussant said, “much of the concern about data quality is related to our own hang-ups as data nerds and what we feel comfortable putting out there for people to use to make decisions. We always say ‘we need more research.’” But here the context is different. “Stakeholders and management want the answer. We need to just put the data out there with some caveats to help them.” One way to offer more context for a dashboard is creating a dashboard report that provides some narrative alongside the visualization. Dashboards should also show trends, not only what has happened already, she said. People need to see trends towards the future so that decisions can be made. It was also pointed out that a dashboard shouldn’t be the only basis for decisions. Like a car dashboard – these data dashboards signal that something is changing but you still need to look under the hood to see what it is. The dashboard should trigger questions – it should be a launch pad for discussion.

13) Organizational culture is a huge part of this process

The internal culture and people’s attitudes towards data are embedded into how an organization operates, noted one Salon participant. This varies depending on the type of organization – an evaluation focused organization vs. a development organization vs. a contractor vs. a humanitarian organization, for example. Outside consultants can help you to build a dashboard, but it will be critical to have someone managing organizational change on the inside who knows the current culture and where the organization is aiming to go with the dashboard process. The process is getting easier, however. Many organizations are thirsty for data now, noted one lead discussant. “Often the research or evaluation team create a dashboard and send it to the management team, and then everyone loves it and wants one. People are ready for it now.”

More resources on data dashboards and visualization.

Special thanks to our lead discussants and to our hosts for this Salon! If you’d like to join our Salon discussions in the future, sign up at the Technology Salon site.

*Salons run under Chatham House Rule, so no attribution has been made in this post.

Read Full Post »

Our April 16th Technology Salon Brooklyn, co-hosted with the Brooklyn Community Foundation (BCF) and AfroLatin@ Project explored the issue of tenant rights within the wider context of structural discrimination. We aimed to think about how new technology and social media might be a tool for helping community organizations to support Brooklyn residents to know their rights and report violations. We were also curious about how better use of data (and ‘big data’) might help housing rights activists and community organizations to more successfully engage residents and advocate for change.

Our lead discussant was David Reiss from Brooklyn Law School, who provided an overview of the wider housing market and challenges in New York City as well as information on some applications that are helping landlords do a better job of keeping properties up to standard. We also heard from Tynesha McHarris (BCF) and Amilcar Priestly (AfroLatin@ Project).

Brooklyn: lots of cool, lots of inequality

Kicking off the Salon, one discussant talked about the essence of Brooklyn. “What do you think of when you hear ‘Brooklyn’?” she asked. “It’s incredibly ‘cool,’ yes. But it’s also incredibly inequitable and there is incredibly inequality, mainly for people of color.” Brooklyn is the hub of New York’s tech industry, yet it’s also where tenants are being displaced, harassed and finding it difficult to live. “We want to see how tech can be used as a tool for, not a tool against,” she said, “how can we support folks to understand, advocate and organize around their rights, how we can use tech to organize in as well as across communities, because these issues don’t just affect some people, they affect all of us who live here.”

She noted that technology is a tool with potential, and donors could be funding projects that use tech to help organize and advocate on tenant rights, but there is insufficient evidence to know how to approach it. To date technology has not really been part of the bigger picture.

Another discussant talked about the housing market as a whole in New York City, citing that there available affordable housing has not kept up with the huge influx of population over the past several years. “Technology will not fix the underlying problem,” he noted. “It can’t expand the supply of affordable housing.” The real potential for technology is more in helping protect the rights of current tenants.

Some examples of how tech is supporting housing rights include applications and portals aimed at improving communications between landlords and tenants, so that problems can be more easily reported by either side, and record is kept of complaints, he commented. Incentives for landlords include free advertising of their units on the site and some reduced legal fees for things like rent stabilization approval. An interesting aspect of these sites is that the data can be analyzed to see where the largest number of complaints are coming from, and in this way patterns can be found and identified. For example, who are the bad landlords? Other sites offer lots of data for those who are interested in purchasing units, and this same type of data could be repurposed and made more accessible for lower-income and less technologically savvy residents.

One participant noted that gentrification and policing are very connected. “As we talk about legal rights and landlord-to-tenant conversations,” she noted, “we need to also bring in aspects of policing and racial justice. These are closely linked.“ As neighborhoods gentrify, newer residents often call for a greater police presence, and this can lead to harassment of long-time residents.

What other roles could technology play in strengthening existing work?

Connecting people and organizations

Lots of community organizations are working on the issues of tenant rights and gentrification, and there is a desire to build a network across these organizations. Tech could help to bring them together and to support stronger advocacy and organization. People don’t always know where they can go for help. One idea was to map organizations in different neighborhoods where people can go for help on housing issues. People also may think that they are the only tenants in a building who are having trouble with a landlord. Improved communication via tech might help let residents know they are not alone and to reduce the fear of reporting and speaking out about housing violations. One idea was to use the new system of NYC neighborhood domains to provide local information, including housing rights and specific information on buildings and their histories.

Transferring tactics from one movement to another

We’ve seen the huge role that mobile video has played in raising awareness on the issue of police violence, noted one discussant. “Technology has become a very powerful tool for communication and accountability, look at the case of Walter Scott (who died at the hands of a volunteer policeman). The young man who filmed Scott’s death knew just what to do. He pointed his camera and captured it. How can we transfer this kind of action over to the housing movement? How can we get people to use their cameras and record housing violations and landlord harassment?”

Offering new, potentially more effective ways to report housing violations

Tech can offer different dissemination channels for different people – for example, in Detroit the elderly are particularly vulnerable to housing violations, said one Salon participant. One organization encourages people to report housing harassment via SMS. They included a call-back option to cater to older people who did not feel comfortable with SMS. Stories are also an important part of campaigns and public awareness, noted another participant. Sandy Storyline created a way to share text plus a photo via SMS for those who wanted to communicate stories but who were offline. This type of application could serve as a way of keeping record of housing violations, when/where they are reported and what the outcomes are.

Tracking housing violations

One way that tech is already helping is by tracking whether public housing buildings have heat and water. Sensors are attached to the buildings, and the information is sent to journalists who then write stories when building violations happen, mentioned one Salon participant. This could be accompanied by text messages out to residents of these buildings to inform them of the status of their building. Knowing that they are not the only ones noticing problems could help residents feel more confident about speaking out and confronting bad landlords. “It’s information that says to someone: ‘this message it not only for you, it’s for everyone in your building, and here is the number you can call to get support or if you fear retribution for reporting.’” Media attention puts pressure on landlords and can help bring violations to light and make people feel safer reporting them.

Encouraging local politicians to get involved

A study in Kenya found that youth tend to bypass local politicians and pay more attention to national government and governance. Similar trends are found in the US where although local political decisions may impact more directly on residents fewer are involved in or aware of local political processes than national ones. Tech could play a role in helping connect residents to local representatives who could take action to support fair housing, address bad landlords, and support longer-term solutions as well. Some local political offices have been very open to integrating technology into their work, said one participant, and these offices might be good places to think about partnering on initiatives that use technology to better connect with their constituencies.

Tracking and predicting trends and population movements and displacement

Mapping and big data sets are providing investors with incredible amounts of information on where to purchase and invest. How can organizations and advocates better use this information, not just to identify movement and displacement and conduct research on it, but also to predict it, prepare for it, and fight it together with residents? How is information that data scientists and research institutes have, as well as open data sets on New York City used by local organizations, some wondered, and where could it be better brought to bear? “Rather than coming up with parallel studies, how can we advocate for more and better open data from New York City on housing?” asked one participant.

Other recommendations

Don’t forget about the legalities of videotaping and sharing

Some people and politicians are pushing to make things like police videotaping illegal. This happened recently in Spain with the so-called “Citizen Security” law that has made it illegal to videotape a police officer in some cases. One discussant mentioned that some US Senators are also trying to restrict the rights of citizens to film police, and that advocates of social justice need to fight to keep these rights to document authorities.

Use the right technology for the audience

One participant noted that you can create great apps with all kinds of data and patterns, but the question is more about who will access and use them, and who is benefiting from them. Wealthy white men and already-privileged people will likely find it very simple to find and use the information and these applications, giving them an advantage in terms of finding good apartments at lower prices, with good landlords. The best way to reach lower income people, he said, as personally experienced from working on political campaigns, is knocking on doors and reaching out personally and directly. “We need to see how to marry community organization and technology.”

Understand the landscape

In order to understand what tech tools might be useful, it we need to understand the communication and technology landscape in which we are working. Though Salon participants mentioned the importance of certain print publications, community radio stations in various languages, and increasing use of smart phones by young people, no one was aware of any current and widespread information on the information and communication habits of residents of Brooklyn that could help to target particular outreach efforts to different groups who were at risk of housing violations.

SMS is not a silver bullet – and trust is key

SMS can be extremely accessible, and there are many examples where it has worked very well. But experience shows that SMS works best where there are already strong networks in place, and trust is hugely important. One participant cautioned, “People need to trust where the text message is coming from. They need to know who is sending the text.” SMS also has limits because it is hyper local. “You won’t find it working across an entire Borough,” said one participant.

Local organizations are key

Along with the issue of trust is the critical component of local organizations. As one participant reminded us, “especially faith-based organizations – temples, churches, mosques. They know everyone in the neighborhood and what’s going on. They tend to know how to walk a fine line on local politics.”

Youth could play a role

Because youth around the world, including in Brooklyn, tend to be up on the latest technology, they could play a role in helping parents and grandparents with housing rights violations, especially in communities where older people are not comfortable with English or where they may fear the police due to undocumented status or other past experiences. One idea was bridging the technology and age gap by engaging young people, who could help older people find out about their rights, legal support services and where to find help. Some research has shown that young people are starting to rely on technology as an institution, said one participant, with technology and online institutions replacing physical ones for many of them.

Be careful about creating demand without adequate response capacity

As with any tech project, creating demand for services and informing people about the existence of those services is often an easier task than building and sustaining the capacity to provide quality support. Any efforts to generate greater demand need to be accompanied by capacity and funding so that people do not become apathetic or feel that they’ve been tricked if they report a violation and do not receive the support they expect or were promised. Previous experiences with service providers or legal institutions will also impact whether people trust these efforts, even if they come through new channels like technology.

Figure out how community organizations and technology partners can work together

An important thing to work out is what a relationship between community organizations and technology partners might look like. “Community organizations don’t need to become technology experts, we could partner and work together on resolving some of these challenges,” said one participant, “but we need to figure out what something like that would look like.” In some cases, community organizations in Brooklyn have low capacity and extremely poor infrastructure due to limited funding, commented one participant. “How can we reach out and engage with them and ask if they are interested in working with tech partners? How can we find out from them what tech would be supportive for them in their work?”

Think about short and long-term efforts

It will be important to look at both supporting residents and community organizations in the immediate term, and thinking about how to use data and information to help address the long term and the wider structural issues that are playing a role in housing rights violations and differential impacts of the housing situation on specific groups, for example, the elderly and people of color. It’s also important to try to address some of the root causes – for example, as one participant asked, “Who is funding predatory landlords? Who are the investors in these vulture funds?”


In conclusion, participants expressed their interest in continuing discussions and a desire for greater participation by community organizations in future Salons. The hope is that the Salon can help to connect community organizations and those in the tech space in order to work together to address some of the issues that Brooklyn residents face.

If you’d like to join us for our next Salon, sign up here.

Many thanks to the Brooklyn Community Foundation for their fabulous hosting and AfroLatin@ Project for helping make the Salon happen!




Read Full Post »

I had a few welcome breaks from my habitual ‘learning’ zone this past month — got away from workshops, conferences, panels, academic reports, meetings, articles and posts for a bit and popped into a little art, comedy and fiction.


Marlene Dumas’ superb exhibit at the Tate Modern in London is called ‘The Image as Burden‘ (it’s on till May 10 so go check it out!). It’s quite relevant for those of us thinking about poverty porn and ethical use of image and narrative in development work, especially Black Drawings. The description of the piece said that she found old photos of Africans, usually taken by anthropologists or colonizers, and where the focus was almost always on black bodies, not on persons, and where individuals were never named. She enlarged the photos and painted close-ups of the faces as portraits, re-focusing on the individuality and humanity of each person.

As I stood there absorbing the wall of faces, it struck me that no amount of ranting and preaching to people about the single story narrative or the way that Africans and the poor are so often stereotyped and filmed and photographed as objects rather than subjects can really bring the point home like this.

Dumas also has a moving piece called Great Men, where she’s done portraits of famous men in history who were gay, using a similar technique of close up painted portraits, here each with a short biography.


Last week, I went to my second stand-up comedy show from America Meet World, where comedians from different countries showcase their craft to US audiences. Trina Das Gupta, who has always been irritated by poverty porn and the aid industry’s single story penchant, runs the production. She started it as a way to lower cultural barriers and introduce the ‘rest of the world’ to Americans. What better way than comedy, she figured. When she told me about her idea a few years ago, I wondered how comedy would translate — they always say humor is cultural, but her strategy is proving to be brilliant. The two shows I hit were hilarious, and I don’t normally follow comedy. The Daily Show is also embracing the idea of a more globalized comedy in the US, with their recent choice of Trevor Noah to replace Jon Stewart. Comedy, when done right, is so good for pointing out absurdities and making you think about yourself and your culture in different ways. (It’s even better when you go out dancing afterwards.)


J. (formerly @talesfromthhood) just put out his latest book, one of the very few in the genre of ‘humanitarian fiction’. This is J’s third novel and he’s firmly settling into the role of writer. In Honor Among Thieves, he introduces readers to likable characters struggling to be ethical in their various roles as development workers. By exploring the challenges and obstacles that people at different levels and in different sides of the industry face, he helps those already inside the industry and those just getting into it to deepen their understandings of the contradictions inherent in the aid system. It would be great reading for some of the journalists and aid critics who like to bash individual aid and development practitioners without understanding the trade-offs they often have to make. The book is entertaining and easy to get through on a plane ride. It critiques the industry but in a more fun and accessible way than articles and posts from academics and journalists and aid critics. (If Honor Among Thieves is too serious, the old fallback ‘Disastrous Passion‘ explores many of the same themes but takes the form of a ‘humanitarian romance novel’, with hilariously over the top sex scenes to break up any serious talk).

We need more art and edutainment

We could classify all these as ‘edutainment’. I looked up the term to see how long ‘edutainment’ has been around. According to Wikipedia, it’s about 50 years.

Since the 1970s, various groups in the United States, the United Kingdom, and Latin America have used edutainment to address such health and social issues as substance abuse, immunization, teenage pregnancy, HIV/AIDS, and cancer.

Parables and fables have been around for quite a bit longer the entry notes (obviously). And of course fairy tales and nursery rhymes sneak advice and warnings inside of clever poems, songs and stories. Some might say that the sacred texts of the world’s major religions are edutainment. But at the risk of offending, I will keep quiet about that.

It’s kind of funny that we (meaning ‘we aid and development people’) like to use edutainment to achieve behavior change with ‘the poor’ but we don’t do nearly enough of it with ourselves and our donor publics.  I, for one, think we need more edutainment. More Fail Fests (comedy plus theater). More satire like Africa for Norway and Tim’s Revolutionary One for One campaign. More shows like The Samaritans and blogs like Stuff Expat Aid Workers Like.

More, more, more!

Read Full Post »

Older Posts »


Get every new post delivered to your Inbox.

Join 894 other followers