The first 72 hours of the SomaliaSpeaks deployment were particularly intense. The purpose of this joint write-up with Al Jazeera and partners is to share some of our early lessons learned in this novel collaboration.  Every deployment teaches us a multitude of lessons, so our partners at Al Jazeera, Souktel and Crowdflower joined us in this effort to share these. We look forward to future collaborations with them as we share this story with you.

This purpose of this pilot project was to let Somalis speak for themselves. For the first time ever, a prominent news organization, Al-jazeera, used crowdsourcing and SMS to let thousands of Somalis express for themselves how the crisis has been effecting their daily lives. More than 4,000 text messages were received within just a few days. Of these, over 1,000 were translated from Somali into English by about 80 translators. The resulting map of Somali voices received over 25,000 page views.

Before reviewing our lessons learned, we first wanted to thank K’naan and Sol for the initial inspiration behind this project. They got in touch with Ushahidi last year because they wanted to use the platform to help amplify Somali voices and show how capable the Somali people are. The initial version of this project was a prototype that was not activated. But thanks to Al- jazeera, Souktel and Crowdflower, we were able to revive the project to help amplify Somali voices in the international media.

Despite being a pilot, the project exemplified valid use cases in the application of pervasive technologies such as the web and mobile phone to news and information gathering. Somalia being a country run down by decades of neglect and war as well being rife with insecurity  provides one of the most challenging operational environments. Within a very short time we were able to curate information traversing through different parts of Somalia. This would have proved futile if not overly expensive or impossible had it been done using traditional news gathering techniques. Furthermore the information collected provides more insight on the realities of life in Somalia.

Projects like this involve a lot of effort and goodwill from the community and a chain is only as strong as its weakest link. Therefore, building a strong community around the project goes a long way to solving problems and mitigating challenges. The community goes beyond the volunteer translators. We saw lots of support from the Somali blogging community, technology and media enthusiasts as well as innovators or innovation centric minds across the globe who all narrated or reported the Somalia Speaks project in their own ways. Having a strong community and inculcating a community ethos in  project operations and goals goes a long way influencing success. Without the community backing and promoting this project, it would have only been another temporary spotlight on Somalia.

The project worked as follows. Al-jazeera editorial selected the following question for interview:

“Al Jazeera would like to know — how has the Somalia Conflict affected your life? Please also include the name of your hometown in the response. Thanks.”

Our colleagues at Souktel distributed the question via text message to 5,000 of their SMS subscribers across Somalia. The responses were then forwarded from Souktel’s SMS platform to a customized Crowdflower micro-tasking platform. There, Somali-speaking volunteers translated and geo-located the text messages which were then manually uploaded to Al-jazeera’s Ushahidi platform.

There are three points worth highlighting in terms of lessons learned:

1. Messaging

While the question that was posed via SMS in no way asked for individuals to reply with their personal names, a small number of responders still added their names; some even added their full names. So these were deleted as quickly as possible. (Note that the numbers posted in the title of initial reports were not phone numbers but an assigned sequential number generated by the Crowdflower plugin). In hindsight, the SMS sent out with the question should have specifically asked that responders not include personal identifiers in their SMS replies.

2. Volunteer translation

While we had recruited a small number of trusted volunteers to translate the incoming text messages using a Crowdflower plugin, a decision was subsequently made to make the call for volunteers public to cope with the 2,500+ SMS replies received. This means that anonymous volunteers could see the original text messages, some of which initially  included personal identifiers. So we immediately reached out to Crowdflower for guidance to take the plug-in offline. We then began to manually delete several dozen text messages inside the Crowdflower plug-in that contained personal identifiers. Our colleagues at Al-jazeera took over this process and set up their own micro-tasking platform, removing all personal identifiers from the text messages awaiting translation and geo-location.

3. Security

One of Ushahidi’s community members tested the platform and identified a search security issue on Friday, December 9th. We quickly fixed this on the deployment. And, we issued a security patch to all deployers. (http://security.ushahidi.com)

In the future, for this type of “The People Speak” project,  we recommend taking the following steps:

1. On large multi-partner deployments: Global organizations work in multiple timezones, so communications plans need to include 24/7 points of contacts for each organization.
2. Text potential interviewees to ask whether they agree to be interviewed and to have their responses made public before sending out the main question.
3. Text those individuals who have consented to being interviewed with the desired question and ask them to include the name of their town but not their personal names.
4. Recruit trusted translation volunteers well in advance and ensure that the micro-tasking translation platform has no personal identifiers.
5. Stagger the launch of the text messages and the live map. That is, start with the SMS broadcast and spend however many days/weeks doing the bulk of the translation with vetted volunteers. The system that holds the raw text messages should obviously be fully secure. When the majority of text messages are processed, launch the live map and gradually add the already translated text messages to grow the map steadily over a period of days/weeks.

At Ushahidi, we’ve also made some plans to help all deployers in our community:

• We’re building a program for privacy and security education for our users in 2012. (Blog posts, webinars, videos and meet-ups.)
• Best practices for security and privacy will be included as essential documentation on our soon to be re-launched wiki.
• Ushahidi is open source and the community is a large part of what makes it work.  We’ll build a security working group focused on our software, but it’ll take your participation to make it work.
• Alongside our partners within the CrisisMappers community, we will participate in a security and privacy working group. This field is growing and collective lessons can only improve each map action.

Crisis mapping and journalism are both in the nascent stages of collaborating on real-time news connecting diaspora and citizens alike. Al-Jazeera is leading the fray in testing and implementing live maps into their fast-moving news cycle toolkit. We are thankful for all their efforts and look forward to further collaboration.

The joint project combines the efforts of multiple partners including Al-Jazeera, Ushahidi, Souktel, Crowdflower, the African Diaspora Institute and the wider Somali Diaspora. The basis of my pitch to Al-jazeera was to let ordinary Somalis speak for themselves by using SMS to crowdsource their opinions on the unfolding crisis. My colleagues at Al-jazeera liked the idea and their editorial team proposed the following question:

Al Jazeera wants to know: how has the conflict of the last few months affected your life? Please include the name of your hometown in your response. Thank you!

So I reached out to my good friend Jacob Korenblum at Souktel. He and I had been discussing different ways we might combine our respective technologies to help in Somalia. Souktel has been working in Somalia and providing various SMS based solutions to several organizations. Jacob had previously mentioned that his team had a 50,000+ member SMS subscriber list. This proved to be key. Earlier this week, the Souktel team sent out the above question in Somali to about 5,000 of their subscribers. An effort was made to try and select geographically disbursed areas.

We’ve since received well over 2,000 text message replies and counting. In order to translate and geolocate these messages, I got in touch my colleagues Vaughn Hester and Lukas Biewald at Crowdflower in San Francisco. Crowdflower uses micro-tasking solutions to process and structure data flows. They were very keen to help and thanks to their support my Ushahidi colleagues Rob Baker and Linda Kamau were able to customize this Crowdflower plugin to translate, categorize and geo-locate incoming text messages:

They also wrote additional software so that text messages from Souktel could be automatically forwarded to the Crowdflower plugin which would then automatically push the processed SMS’s to a live Ushahidi map hosted by Al-jazeera. While the software development was moving forward, I connected  with colleagues from the Somali American Student Association who expressed an interest in supporting this project. Thanks to them and other members of the Somali Diaspora, hundreds of Somali voices were translated and shared on Al-jazeera’s public Ushahidi map of Somalia within days. But we still need lots of help. So if you speak Somali and English, then simply follow this link.

I wanted this project to serve as a two-way conversation, however, not just a one-way information flow from Somalia to the world. Every report  that gets mapped on an Ushahidi platform is linked to public discussion forum where readers can respond and share their views on said report. So I suggested that Al-jazeera invite their viewers/readers to comment on the text messages directly. The next step will be for Al-jazeera’s editorial team to select some of the most compelling and interesting comments and to text these back to the senders of the original text messages in Somalia. This two-way flow of information can be iterated and scaled given that the technologies and workflows are already in place.

In sum, the purpose of this project is to catalyze global media attention on Somalia by letting Somali voices take center stage—voices that are otherwise not heard in the international, mainstream media. If journalists are not going to speak about Somalia, then this project  invites Somalis speak to the world themselves. The project highlights  these voices on a live, public map for the world to bear witness and engage in a global conversation with people of Somalia, a conversation in which Somalis and the Diaspora are themselves at the centerfold. It is my sincere hope that advocacy and lobby group will be able to leverage the content generated by this project to redouble their efforts in response to the escalating crisis in Somalia.

I very much hope to see this type of approach used again in Somalia and elsewhere. It is fully inline with the motivations that inspired the launch of the first Ushahidi platform almost 4 years ago today: collective witnessing. Indeed, I am often reminded of what my friend Anand Giridharadas of the New York Times wrote last year vis-a-vis Ushahidi. To paraphrase:

They used to say that history is written by the victors. But today, before the victors win, if they win, there is a chance to scream out with a text message, a text message that will not vanish, a text message that will remain immortalized on a map for the world to bear witness. What would we know about what passed between Turks and Armenians, Germans and Jews, Hutus and Tutsis, if every one of them had had the chance, before the darkness, to declare for all time:

“I was here, and this is what happened to me”?

Use #SomaliaSpeaks to witness the project on Twitter.

I want to specifically thank the following individuals who put an incredible amount of time and effort (most pro bono) to make this project happen: Robert Baker, Linda Kamau, Michael Moszczynski, Katie Highet, Jacob Korenblum, Vaughn Hester, Mohammed Dini, Hamza Haadoow, Andrew Jawitz and of course the excellent Al Jazeera team in Doha. Thank you all for going above and beyond to make this happen.

]]> http://blog.ushahidi.com/index.php/2011/12/08/somalia-speaks/feed/ 29 http://blog.ushahidi.com/index.php/2010/02/07/sms-turks/ http://blog.ushahidi.com/index.php/2010/02/07/sms-turks/#comments Sun, 07 Feb 2010 21:58:27 +0000 Brian Herbert http://blog.ushahidi.com/?p=1418 If you’ve been reading along on the Ushahidi Blog, you will know that the coordination efforts around the Haitian Earthquake have been nothing short of amazing. The students and volunteers at the Fletcher School Situation Room, the translation volunteers on the Mission 4636 project, the teams and staff of Digicel, Comcel, Energy for Opportunity, FrontlineSMS, InSTEDD, Sahana, Cartika Hosting, the US State Department, almost all branches of the US Military providing humanitarian response and a list of individuals and organizations that could honestly go on forever, have come together in an unprecedented way to work together to help solve problems on the ground and to get information out to any and all interested parties.

My role in all of this started shortly after the Ushahidi-Haiti instance was up and running, providing technical support and new, rapid development on the instance as needs arose. Virtually all of the core developers were working around the clock making sure critical bugs and new features were taken care of, as well as making sure the servers were running smoothly.

The home of the future redevlopment of SMS Turks on github.

While everyone was in full gear working on the website, we were able to secure the 4636 short code with the help of Josh Nesbit of FrontlineSMS, Digicel and Comcel. We just had one problem, the stakeholders who were going to be digesting these messages and passing them along to the appropriate organizations spoke English and some French. Messages being sent from Haitians on the ground would be coming through primarily in Haitian Kreyol, which would have made it nearly impossible to categorize, map and respond. So, my focus shifted towards the short code effort. With the help of InSTEDD donating server space and Robert Munro handling volunteer feedback, I was able to write a system at 4636.ushahidi.com that would allow translation, categorization and basic geocoding of all the messages that came in. I’ve coined this project, “SMS Turks.”

In crisis situations, it’s always better to use systems that have been tested thoroughly that can scale well. Since SMS Turks was literally put into production the day it was built, there were bound to be issues. Also, volunteers can only put in 12 hour days translating text messages for so long. CrowdFlower graciously offered their services to pipe the messages through their system, handling the technical aspects at no cost to Ushahidi. Over time, as volunteers go back to their day jobs, Samasource will be providing Haitian’s paid opportunities to process the messages as they are coming in, allowing us to put money into the Haitian economy.

The SMS Turks system will be entirely rewritten from the ground up as an Ushahidi project. It will be easily pluggable into Ushahidi, as well as produce feeds that should work with virtually any other open system.