Androids running SMSSync at Ushahidi Liberia

These days technology just works. There’s no magic and very little rocket science; put in some 1s and 0s and get out 1s and 0s. The days are gone when bugs, actual insects, would chew their way through the computers wire and cause mayhem.

Now some of you right now are saying, “Yeah, but I still can’t figure out how to make my Facebook profile private.” That’s a user interface issue; the underlying technology is working perfectly. The layer that exposes that technology to you may be poorly designed, but that’s not what we’re here to discuss.

Yes technology hums along gloriously – until certain assumptions no longer hold true. Such assumptions usually include constant electricity and Internet, two things that are not constant in Liberia. Thus what seems like a perfect combination – SMS technology and Liberia – has a few obstacles to overcome before riding off into the sunset happily ever after.

For us at Ushahidi Liberia, we wanted SMS and Liberia hit it off. SMS is a great way for people on the ground to send in reports of what’s really happening. All you need is a phone and a cell phone signal. Most places in the world have cell phone coverage, and an increasing number of people have cell phones. The Ushahidi platform even has built-in support for SMS because it has worked well in other deployments. So let’s get to know more about the compatibility of this couple in particular – SMS and Liberia.

SMS

Simple Messaging Service is a GSM standard that uses extra bandwidth in the signaling path that controls call flow. The signaling path is used to tell a cell phone that a new call is coming in, that the call has been hung up, the number of the incoming call and so forth. Since the signaling path isn’t used when there is no phone call, phone companies realized they could add a messaging service on top of this unused signaling path – and SMS was born. Because the signaling path is only intended for short messages like, “incoming call +231-6-555-343”, SMS messages can only be 160 characters long.

Liberia

Liberia is a small country in West Africa still recovering from a civil war that devastated the nation’s infrastructure.  Despite the lack of power lines, power stations and any kind of hard-wired telecommunications infrastructure, Liberia now has a relatively robust cell phone network. While less than 1% of Liberians have access to the Internet, at least 20% own cell phones and even more have access to shared phones. SMS seems like a natural choice for Liberians to send messages to our Ushahidi Liberia sites, given that even the cheapest cell phones available in-country support SMS.

Half a powerline: Liberia's damaged infrastructure

The Trials of Matchmaking

At this point, things look good for our two lovebirds. A simple global standard for sending short messages and a country where the telecommunications infrastructure is best equipped to handle short messages. In fact, such a partnership has worked so well in other countries that the wonderful people at kiwanja.net created FrontlineSMS. FrontlineSMS is a program that turns your average laptop and a cell phone into an SMS gateway for bulk messaging and routing incoming SMS to the Internet. By connecting a cell phone to a computer via a data cable, FrontlineSMS can intercept incoming SMS and automatically send them to a website of your choosing – an ideal matchmaker for SMS and Liberia.

However, it is here we encountered our first obstacle. Computers need power. You can get around this requirement for a few hours by using a laptop, but the problem persists. Ushahidi Liberia’s first office used a diesel generator for electricity, like most Liberian workplaces that do not have access to a power grid. Diesel isn’t cheap in Liberia, so the generator only ran from 9am to 7pm; that leaves 14 hours without power.  In other words, 14 hours without the ability to send and receive SMS via a desktop SMS gateway.  Because many of the reports sent to our Liberia instances about conflict and instability, we could not afford to be operational for only 10 hours a day.

The second obstacle was an unreliable Internet connection. In Liberia, all Internet connections are via satellite – far slower and more expensive than fiber optic connections. Ushahidi Liberia’s first ISP leased a satellite Internet connection comparable to a slow DSL connection in the US – comparable until the ISP splits that connection among all of its customers, who then split their slice of the connection amongst all the users in the office/home.  During peak working hours as many as 500 people were using that one Internet connection, causing it to drop out when bandwidth was exceeded, and causing any user to grind their teeth in frustration when it was working but oh so slow.

When the Internet dropped out in our office and FrontlineSMS received an SMS to forward to Ushahidi servers, the sending would fail and FrontlineSMS would drop the message without resending. At that time, FrontlineSMS did not notify the sender or the receiver that a message failed to send, so it could be days before the person running the FrontlineSMS instance might realize that half the sent messages were missing. FrontlineSMS, like many platforms, is designed with the assumption that the Internet works.  Granted, we were using FrontlineSMS as an SMS gateway to the Internet and not solely as a bulk messaging system, so our needs were specific.

SMSSync to the Rescue

Considering these glitches, our couple might not be as compatible as we thought – that is until SMSSync came along. SMSSync is an Android app written by the Ushahidi team that replaces the computer as the intermediary and runs the SMS gateway program on the phone itself. Since Android phones can connect to the Internet via WiFi and GPRS, they can receive an incoming SMS and then send it out over the Internet all by themselves. This solved our first problem of power. Phones can easily run 14 hours, or a couple of days, without recharge; that’s what they’re made to do. But we still have the issue of unreliable Internet.

SMSSync Android app

To address the Internet, our team worked closely with SMSSync’s creator, Henry Addo, to incorporate a resend function that repeatedly tries sending a message until it is received by the target URL. Now messages can be received by SMSSync even when the Internet is out and they stay in a holding pattern until the connection returns.

At this point it seems like everything is going to work out for SMS and Liberia, but not so fast.

There’s still one more assumption suggested earlier – that humans will interact with technology correctly. Even the most well-intentioned, experienced user will occasionally get it wrong. The civil war held up the arrival of the latest technologies, and as a result many Liberians don’t have extensive experience with technology.  We noticed that every so often we’d receive a blank message from our users in the field; someone probably hit send prematurely. At first we didn’t pay much attention to this, but then we noticed that SMSSync would stop sending messages after receiving blanks.

After a lot of hair-pulling, we realized the glitch was related to the Ushahidi platform itself. It was programmed to reject blank SMSs as erroneous. SMSSync would try to forward the blank SMS to Ushahidi, Ushahidi would reject it, SMSSync would wait 5 minutes and try again, meanwhile all the other messages were waiting in line. We reprogrammed the Ushahidi platoform to accept all messages, blank or otherwise. Messages that appeared to be errors would be marked as such for the users of the Ushahidi platform to decide what to do with them.

We also had a similar problem with promotional SMS from the cell phone companies. They’d send out things like, “Talk free this Saturday” and often these messages wouldn’t be from numbers like, “06-555-123”, but rather from “winBig” or “LonestarCell.” Again, we didn’t think much of this, but SMSSync stopped working shortly after receiving these messages. It turns out that the Ushahidi platform is also set to reject SMS from numbers that aren’t numbers, sending an app like SMSSync into a never-ending loop. We also fixed this in the Ushahidi platform.

At this point we’ve accounted for assumptions about electricity, Internet and human users. The final assumption we had to overcome is that the technology will always, from now to eternity, until you tell it otherwise, do what you want. But here’s a twist: in an effort to save battery power, Android phones are programmed to turn off their WiFi radios after a certain period of inactivity. Thus SMSSync would work brilliantly for awhile, but then stop forwarding messages for no apparent reason. We’d look at the phone, see the unsent SMS, and since we were now using the phone the WiFi would come back on and mysteriously work. Much to our embarrassment, this also took a long time to figure out.

It’s the Simple Things

At this point I’d like to make it clear that 99% of the time it’s the simple things that get in the way. It wasn’t some small manufacturing defect in our phones, it wasn’t a rogue bit of code deep in SMSSync, it was just a simple feature of the phones that, when they aren’t working as SMS gateways, works to the user’s advantage.

KeepScreen Android app

We fixed the WiFi automatic sleep by installing KeepScreen on the phones. KeepScreen is a free Andoid App that just keeps the screen on all the time. By keeping the screen on, and making the Android think the user is still using the phone, the WiFi also never goes off.

Now our phones work perfectly, day and night, through power and blackouts, high bandwidth bliss and connection timeouts, to send SMSs from our users on the ground to our servers high in the Internet’s metaphorical cloud. And after much hardship, Liberia, a beautiful country, and SMS, a messaging protocol of elegant simplicity, are together at last.

In a nutshell, load testing is testing a website to see how many users can access the site at any given point in time. We ran “ramp up” tests that simulates the number of users from 10 users to N users in 10 steps. The first sub-test runs against 10 users and the last sub-test runs against N users. The 8 tests between the two slowly ramp up the number of users to N. If the users timeout, the test stops. Finding out how many users causes timeouts is the technical limitation of the number of users that can access the site. However, it should be kept in mind that users will only wait so long for a site to load.

Server Configuration & Testing Process

The server was configured to closely mimic a typical setup you could find from most web hosts. Our server had 2,048 MB of RAM, 80GB of hard disk running Apache 2.2.3. We used a real world data set from the Haiti deployment. We used the default configuration for one set of tests and we used the default configuration with CDN configured using Rackspace Cloud Files.

We ran load tests on the homepage alone. Load Impact allows us to test the site in any different number of ways, including recording a sequence of clicks moving around the site, but we wanted to focus on simple homepage loading.

Load Testing Results

Using a CDN greatly improves performance

For our first test running against the default, non CDN enabled configuration of the platform, we found that the maximum concurrent users was low. This setup can sustain about 100 concurrent users before load times get over 10 seconds. At this point, most users will have already left the site. For a casual deployment, this shouldn’t be much of an issue. However, in the case of deployments that quickly gain in popularity, something needs to be done to improve scalability.

When we ran the second test with the CDN enabled, we found that we were able to take on many more concurrent users. By simply offloading static resources, you can have 4 times as many concurrent users under 10 seconds of load time. As you can see on the chart above, it drastically increases the number of concurrent users before we hit a technical limit of concurrent users.

Use a CDN

If you want to use a CDN, it’s quite simple. Just sign up for one of the CDN services like Rackspace Cloud Files or Amazon CloudFront. Upload the contents of your media directory there and then add the URL to your files in applications/config/cache.php. You will need the latest version of the Ushahidi Platform to do this.

I’ve spent the past week at the iLab in Liberia and got what I came for: an updated reality check on the limitations of technology adoption in developing countries. Below are some of the assumptions that I took for granted. They’re perfectly obvious in hindsight and I’m annoyed at myself for not having realized their obviousness sooner. I’d be very interested in hearing from others about these and reading their lists. This need not be limited to one particular sector like ICT for Development (ICT4D) or Mobile Health (mHealth). Many of these assumptions have repercussions across multiple disciplines.

The following examples come from conversations with my colleague Kate Cummings who directs Ushahidi Liberia and the iLab here in Monrovia. She and her truly outstanding team—Kpetermeni Siakor, Carter Draper, Luther Jeke and Anthony Kamah—spearheaded a number of excellent training workshops over the past few days. At one point we began discussing the reasons for the limited use of SMS in Liberia. There are the usual and obvious reasons. But the one hurdle I had not expected to hear was Nokia’s predictive text functionality. This feature is incredibly helpful since the mobile phone basically guesses which words you’re trying to write so you don’t have to type every single letter.

But as soon as she pointed out how confusing this can be, I immediately understood what she meant. If I had never seen or been warned about this feature before, I’d honestly think the phone was broken. It would really be impossible to type with. I’d get frustrated and give up (the tiny screen further adds to the frustration). And if I was new to mobile phones, it wouldn’t be obvious how to switch that feature off either. (There are several tutorials online on how to use the predictive text feature and how to turn it off, which clearly proves they’re not intuitive).

In one of the training workshops we just had, I was explaining what Walking Papers was about and how it might be useful in Liberia. So I showed the example below and continued talking. But Kate jumped in and asked participants: “What do you see in this picture? Do you see the trees, the little roads?” She pointed at the features as she described the individual shapes. This is when it finally dawned on me that there is absolutely nothing inherently intuitive about satellite images. Most people on this planet have not been on an airplane or a tall building. So why would a bird’s eye view of their village be anything remotely recognizable? I really kicked myself on that one. So I’ll write it again: there is nothing intuitive about satellite imagery. Nor is there anything intuitive about GPS and the existence of a latitude and longitude coordinate system.

Kate went on to explain that this kind of picture is what you would see if you were flying high like a bird. That was the way I should have introduced the image but I had taken it completely for granted that satellite imagery was self-explanatory when it simply isn’t. I really kicked myself on that one. In further conversations with Kate, she explained that they too had made that assumption early on when trying to introduce the in’s and out’s of the Ushahidi platform. They quickly realized that they had to rethink their approach and decided to provide introductory courses on Google Maps instead.

More wrong assumptions revealed themselves during these courses. For example, the “+” and “-” markers on the map are not intuitive either nor is the concept of zooming in and out. How are you supposed to understand that pressing these buttons still shows the same map but at a different scale and not an entirely different picture instead? Again, when I took a moment to think about this, I realized again how completely confusing that could be. And again I kicked myself. But contrast this to an entirely different setting, San Francisco, where some friends recently told me how their five year old went up to a framed picture in their living room and started pinching at it with his fingers, the exact same gestures one would use on an iPhone to zoom in and out of a picture. “Broken, broken” is all the five year old said after that disappointing experience.

The final example actually comes from Haiti where my colleague Chrissy Martin is one of the main drivers behind the Digicel Group’s mobile banking efforts in the country. There were of course a number of expected challenges on the road to launching Haiti’s first successful mobile banking service, TchoTcho Mobile. The hurdle that I had not expected, however, had to do with the pin code. To use the service, you would enter your own personal pin number on your mobile phone in order to access your account. Seems perfectly straight forward. But it really isn’t.

The concept of a pin number is one that many of us take completely for granted. But the idea is often foreign to many would-be users of mobile banking services and not just in Haiti. Just think about it: all one has to do to access my money is to simply enter four numbers on my phone. That does genuinely sound crazy to me at a certain level. Granted, if you guess the pin wrong three times, the phone gets blocked and you have to call TchoTcho’s customer service. But still, I can understand the initial hesitation that many users had. When I asked Chrissy how they overcame the hurdle, her answer was simply this: training. It takes time for users to begin trusting a completely new technology.

So those are some of the assumptions I’ve gotten wrong. I’d be grateful if readers could share theirs as there must be plenty of other assumptions I’m making which don’t fit reality. Incidentally, I realize that emerging economies vary widely in technology diffusion and adoption—not to mention sub-nationally as well. This is why having the iLab in Liberia is so important. Identifying which assumptions are wrong in more challenging environments is really important if our goal is to use technology to help contribute meaningfully to a community’s empowerment, development and independence.

Moving out of our one-room office, with its questionable structural integrity, was a good first step.  The picture below, taken when our AC was replaced, gives you a sense of the place.

Ushahidi Liberia's original office

After searching the better half of Monrovia, I found an apartment building still under construction that promised to have everything we needed. We got a good deal because there were nothing but walls and a roof when we moved in; over the following month, the AC, power, and security were fully in place and we had our dedicated VSAT up and running.

New office's future tech lab

The building's wiring

In the meantime, our staff grew from two ex-pats to an additional five Liberians, several of whom are the best and the brightest in Liberia’s IT scene (no exaggeration they are phenomenal), and we now had the space to accommodate them.

Our shared office

Our dedicated VSAT is a modest 384/128 kbps (C-band), but I tell you that is lightning speed in Liberia (and it’s pretty cool to have a dish that big).  The building runs on an industrial generator and has a backup so we’ve got power at all times.  Just in case, we have UPS (Uninterruptible Power Supply) units plugged into our VSAT modem, router and the desktops so they don’t blink out when there’s a power glitch.

Our very own VSAT

In our new space, we can breeze through Ushahidi instances that used to take minutes to upload at the old office.  But as of March, we still didn’t have dedicated equipment for partners – we just had the connectivity and an empty room we called “the lab”.   So we began to purchase computers locally. Always a gamble.  We started out with Dell Inspiron desktops, a hefty 1,200 USD in Monrovia (the same desktop is about 640 USD in the States).  After buying three, we realized that it might be a better move to get twice as many computers abroad for the price of one locally.  This requires importing.  If that word alone doesn’t scare you, suffice it to say bringing tech equipment into Liberia is…complicated.  Organizations here have a whole staff position if not team dedicated to bringing in equipment, and thus far we acquired our Stateside equipment through friends’ suitcases on trips back and forth.  With no importing experience and little more than a dose of determination, I ordered the computers anyway; my trusty mother packed up the eight Lenovo ThinkPad SL510s with FedEx, and off they went.

“You know they’re going to have to pay a large import tax just to get these,” the FedEx fellow said after charging us an arm-and-a-leg for sending the two 40lb packages.  The cost of getting the computers over to Liberia was still less than buying half as many desktops in-country, but I dreaded the inevitable red tape. I had one advantage on my side: our organization’s accreditation in Liberia. With this, we were allowed to get imported items tax-free if they were for educational purposes.  When I got word that the two packages had arrived at the airport in Monrovia, I went to FedEx with accreditation papers in hand.  If we wanted to take advantage of our tax-exemption, FedEx said, the papers weren’t enough; we had to go to the Ministry of Finance and get some countless number of signatures from every higher-up for duty-free permission.  I asked our Director and Deputy Director of IT to make a trip to the Ministry and get the necessary signatures. One of them had worked at the Ministry before, so I thought we had this in the bag.

Instead, the Deputy Minister of Revenue said the approval process for first-time importing would take days.  We didn’t have days; the computers were baking in an airport hangar.  Another option: the Commissioner of Customs could submit a special request to have the packages released quickly.  Both options required formal letters of request, along with a copy of Ushahidi Liberia’s rent check for proof of existence in-country.  I thought the accreditation certificate demonstrated our presence in Liberia, but okay: my team came back, I wrote up the letters, and we printed out the scanned rent check.

Our team returned, with multiple copies of the letter just in case; assistant clerks took the documents and said come back in a couple days.  Two days later, and the documents hadn’t budged; it was going to be another couple days, said the clerks, who indicated that some “cold water” might speed up the process (code for bribe).  When I got word of this, I’d had enough; surely, my Liberian colleagues are more capable of navigating these waters than I, but I was not prepared to lose our computers to negligence and heat.  On top of the mounting obstacles, the Deputy Minister was apparently furious that our landlord was charging so much for rent and the Ministry wasn’t aware of it; word was she had called for an investigation.  Great.

I put on my best power suit and set out for the Ministry.  What ensued can be summarized as follows:

• Deputy Minister’s office: Me – here are the letters again, we need these packages, I am not moving until we get permission (please).  I am ignored for a couple hours until Asst. Deputy comes in and I make a fuss (politely). I’m sent to unnamed office in basement
• Unnamed office in basement: lots of questions about the packages. Details typed up on 1950s typewriter.  Take this to the Asst. Deputy to sign and you’re set. Back to Deputy
• Deputy’s office: clerk looks at the paperwork and puts it down on the desk. “Come back tomorrow.” I say no, all we need is one signature. He doesn’t care. I stand resolute in front of his desk and promise not to move until I have the signature. He says I have to sit down for the process to begin. On the contrary, I will stand. Clerk reluctantly tells me the location of the Asst. Deputy.  I go one door down the hall, find the Asst. Deputy, and he signs

After 5 hours at the Ministry, I made it to FedEx with the documents just before they left for the airport. By 8pm that night we had our computers, tax- and bribe-free.  We set them up then and there, tested out each one, and took a moment to breathe in that new computer smell.

FedEx Delivery

Our IT Director marveled, “I’ve never seen a computer this new before.”  And all eight worked, without a hitch.  Whew.  Now we can start the real work. Stay tuned for the latest on iLab Liberia in our next post.

That new computer smell

Micz Flor of Sourcefabric

This is a guest blogpost from our developer community, written by Micz Flor.
Micz is a media developer, writer and project manager based in Berlin. Since 1995 he has been knee-deep in Internet, organising a variety of events, net projects, magazines and temporary media labs. In 2000 Micz began work as a media developer for Eastern Europe and Asia at the Media Development Loan Fund (MDLF), where he met his fellow Sourcefabric co-founders.

My introduction into developing for Ushahidi was accidental. While I was developing templates explaining the Newscoop CMS mapping feature, I was looking for a valid KML-feed. Our community manager Jakub Górnicki sent me an XML-file from his recent Ushahidi crisis map covering “Heavy Snow in Poland”. He had used this as a layer on the map. The same day, I started abusing Ushahidi layers as a KML-feed validator (I know, that’s not what it was developed for…). Once we put two and two together, the Ushahidi Cooker was born. Ushahidi layers is a really great feature which (in my opinion) is generally underused.

What does the Ushahidi Cooker do?

The Ushahidi Cooker provides a directory of maps and KML-feeds. Or to be more precise: a tool to easily create and manage libraries of maps. These maps are stored by categories and cities, like “Hospitals in Berlin” or “Airports in Indonesia” – anything you like. These maps come with a valid KML feed. And this feed can be called into a Ushahidi map as a layer, providing additional content on the map for the users of your site. Below you can see a video illustrating the process of creating an Ushahidi layer from Newscoop’s Ushahidi Cooker as well as a link to a demo site where you can try the KML-feeds.

Why is this useful?

1.
Using Ushahidi Cooker means that your map is really your map. It is not a Google Map, or in other words: Google’s map. Similar to Ushahidi, Newscoop provides different ways to render your maps. You don’t have to lock your data into a map provider.

2.
With the Ushahidi Cooker you can prepare maps in the background – and call them into different Ushahidi installations when needed. “Bridges in Iceland” could be of interest on a tourist map as well as a crisis map after a volcano eruption. With Newscoop, you can make a copy of any map you have in stock and then make alterations, give it a new name and use it in a new context.

3.
Newscoop is a fully fledged and open CMS for professional journalism. Tying Newscoop into your Ushahidi map gives you the tools in the background to manage articles and other content around the subject of your map.

Try it out and help improve the Ushahidi Cooker

I would be interested to see this tested in a real life situation. Try it out and tell us what you think. If you need it for a crisis situation, we can also help you out with support. If you are interested, contact our community manager Jakub Górnicki (jakub.gornicki at sourcefabric dot org).

Having been “in the field” as a media developer myself, I know that tools which seem to work at first sight only really work once you broke them a few times – and improve them on the way. The Ushahidi Cooker and the Newscoop CMS are free and open source.

The video tutorials show the Ushahidi Cooker in action. Covering the entire process, the installation in Newscoop, creating maps and adding layers in Ushahidi, the total running time is roughly 17 minutes, so it is really very straight forward.

All Video Tutorials:

• Creating Ushahidi layers with Newscoop’s Ushahidi Cooker

(3:23)

• Creating maps with the Ushahidi Cooker template pack (6:56)
• Installing the Ushahidi Cooker template pack in Newscoop (6:20)

Ushahidi Cooker test install
http://ushahidicooker.templates.sourcefabric.org
More information is available on the help page:

Community / Newscoop support
http://forum.sourcefabric.org

Downloads
Download Newscoop
http://www.sourcefabric.org/en/products/newscoop_download/

Download the Ushahidi Cooker for Newscoop
http://www.sourcefabric.org/en/products/newscoop_templates/

Liberia 2011 is an Ushahidi instance dedicated to tracking the country's presidential election

January 6^th marked the beginning of voter registration in Liberia, as well as the nationwide deployment of civic voter education activities, registration monitors, and widespread reports of registrant bribing and illegal registration.  Our team realized that, in order for an Ushahidi instance to be useful in this setting, the instance must catch the attention of many different election-related actors with a variety of roles and interests regarding the election process.  And once we have the attention of these actors, the instance must be accessible and manageable by these multiple users.  Hence, the “Simple Groups” plugin. **Note: in the initial months of the election process, we are not yet targeting ‘the crowd’ but rather organizations that serve an advocacy or monitoring role for the election.

Some of the groups contributing reports to the election instance

With Simple Groups, multiple organizations composed of several unique users can now have private admin access to the same Ushahidi instance. Each group has a private admin page that contains their own messages, reports, and admin map.  This setup keeps data centralized on the instance’s homepage while allowing groups to keep their reports and messages private until they are ok’d for the public map. In short, a plugin for data privacy with a collaborative twist – shared hompeage, private back doors.

An example of a group's private admin page for the election instance. The above page, created on our Demo site for testing purposes, shows the basics of a group page: access to messages, approved and unapproved reports, as well as the Admin Map (found under the Map tab)

Here are some features we’ve created for groups:

• Whitelisted phone numbers – managers of the shared instance (currently limited to our consulting team) have the ability to link trusted sources’ phone numbers to particular groups.  When a message is received from that number, it is automatically forwarded to that group’s private admin page.  Each message from a whitelisted number appears on the group’s messages page with the name, organization and number of the sender listed.  To date, over 135 phone numbers belonging to 30 different reporting organizations have been whitelisted as trusted election reporters.

Whitelisting feature for groups' vetted sources

• Tagged reports – each group’s report is tagged with the organization’s logo so other groups and all viewers can quickly identify the source that has approved and determined the verification status of the report

A report on the election instance tagged by a participating group

• Admin Map plugin– we anticipated that many groups will want to keep certain reports private, if only temporarily, due to their sensitive content. While these unapproved reports are accessible in list form on a group’s admin page, there used to be no way to visualize these unapproved reports on the map. We created the Admin Map on each group’s admin page that shows all unapproved reports as black. This Admin Map provides each group with a view of their reports and their reports only; this way, groups can choose to view their reports exclusively, or to see them with all groups’ reports on the shared homepage

The Admin Map allows each group to privately visualize all its reports, approved and unapproved

• A few more features of the Admin Map:
  • Multiple categories can be compared at once using the Boolean functions OR and AND.  Users can see all the reports that were categorized as A or B, as well as those reports that were categorized as A and B. In addition, when categories A and B are selected using AND, the colors of the two categories blend to uniquely represent those reports. These logical operators give groups a new way to look for trends in their data
  • Many of our partners feel confined by the 4×4” size of the homepage map and spend long minutes with the slow Internet connection zooming in to get a closer look. The Admin Map uses more of the screen and makes the Liberia map easier to navigate
  • The user interface has been modified to make categories easier to explore. Each category with subcategories now has a “+” sign next to it so users do not have to spend valuable Internet time clicking each category to see if it has further divisions. The timeline and category filters may now be positioned on the screen according to the user’s liking
  • All of these features have also been added to the instance’s homepage (all except groups’ unapproved reports).  Click on the “Big Map” button at the top of the election homepage or the “view full map” button just above the map; both will open a map with nearly all the same features as Admin Map

• Location Highlighter – When our partners are turning messages into reports, it can sometimes be difficult to find the message’s location on the map.  To expedite this process, the Location Highlighter tool outlines Liberia’s counties and districts on the map to narrow the scope of the search. This way, when users are looking for a town they can’t find with a quick map scan, but they know the appropriate county and district, they can turn on the Highlighter and zero in on a specific county, district, and even clan. Because most of our groups work with low bandwidth, the Highlighter’s clear parameters allow admin users to spend less time clicking and dragging across the map and more time searching a clearly defined area.  This addition has been especially helpful since Google Maps does not show the districts of Liberia and the counties are not labeled

The highlighting tool identifies the boundaries for a location down to the county, district, and even clan level

Just below the Highlighter tool is the “Find Location” feature.  This has long been a part of the Ushahidi platform, however we’ve found in Liberia that our first search results often landed in India and Ohio rather than Liberia’s interior.  With a few tweaks, the Find Location feature now searches both Google and GeoNames for the town or city listed.  We have also added the ability to set a bounding box so that only results from inside that box (in this case, Liberia) will appear

Group users also found that it took some time, due again to low bandwidth, to get the results of a town search; to expedite this process, results are now set to cache locally so that after the first few searches the data will be stored on the site for faster load time.  We’ve also set the results to show all matches, not just the first, so the user is ultimately the one who chooses the appropriate match.  These changes have also been made to the public reporting form on the homepage’s “Submit a Report”. The Highlighter is a plugin and Find Location is 95% plugin – the only thing that’s different is that Find Location has to operate with the Liberia theme and the Highlighter plugin installed. You can play with these latest location features on our Demo site without worrying about submitting a real report

The "Find Location" section of the plugin provides several options for locating a town or village by its name alone

• End-time plugin – many messages from the field are reports of voter education activities that span several hours and sometimes days.  In order to capture the full length of the reported event, we have added an end-time option for users so the end date and time of day can be listed in the report
• Forwarding messages - if a message comes in from a non-whitelisted number, it is still possible for the instance’s managers to assign it to a particular group.  If the message’s contents indicate the affiliation of the reporter (his/her organization), the message can be forwarded to the appropriate group.  On that group’s private backend, it appears as if it was sent directly to them and not forwarded on from an intermediary

• Reminder cards – while this is not an online feature, it has significantly improved the quality of the messages received.  Each group that requests participation in the election instance receives trainings from our team for office staff (in how to turn messages into reports) and another for field staff and affiliated reporters (in how to report to the election instance).  During the training, a small reminder card is given to each field reporter (the size of a business card) that summarizes the basic steps of reporting as well as what bases to cover in each message. Because texting is still quite new to many of our partners’ staff, especially reporting information via text, we found that training in how to report via text was a must. Presently, more than 60 field reporters have received direct training in using this card, and we anticipate another 60-70 will be trained in the next few months

The front and back of the election reporting card: the size of a business card, this is handed out to field reporters as a reminder of what to include in a message

SMSsync - The election instance has a dedicated shortcode, 2011, that is free with 3 of Liberia’s 5 operators. Our team previously used FrontlineSMS as our SMS gateway; however, because we have a shortcode SIM from each operator and only one extra computer available, we had to run multiple FrontlineSMS instances on that computer – something the program was not designed to support.  We started to consider other options, particularly ones that would not require that a computer be on 24/7 (given the erratic nature of the country’s electricity).  In cooperation with Ushahidi’s talented Henry Addo, we tested downgraded versions of SMSsync to work with android phones available in Liberia.  After just one week, we made the switch: now our SMS gateway operation has downsized and simplified to a mere 3 androids, each with a dedicated 2011 SIM, using GPRS Internet connection to sync messages to the election instance, and only needing a power source for a few hours every other day.

With all of these additions and new technologies, we have kept our user audience small to start.  Voter registration, which officially ended February 12th, was a great testing ground for the new and improving election instance.  We trained a handful of election-focused organizations and so far we have received 325 texts and 150 approved reports on the shared homepage.  The majority of our current reports document voter education activities more than illegal happenings at registration sites or citizens’ feedback about the process; and of those latter reports that do exist, many are privatized by groups because they are still determining what they want to share via this platform.

Our team would love to see a greater variety of messages coming in, reporting on not just organizational activities but also on the many successes and challenges citizens experience around registration.  But we have to think in baby steps.  After several months in-country, I am starting to feel that any info sharing is good info sharing in Liberia.  With each stage of the election process we plan to take another step – spreading the word about the shortcode to the public, bringing on more groups that collect a variety of information, establishing a tech center with reliable power and Internet so groups have a mapping hub at their disposal.  Step by step, it will certainly be a very interesting year.

P.S. to access any of the plugins mentioned here, follow these links:

• http://apps.ushahidi.com/p/endtime/

• http://apps.ushahidi.com/p/findlocation/

• http://apps.ushahidi.com/p/locationhighlight/

• http://apps.ushahidi.com/p/simplegroups/

In the aftermath of some of the recent disasters we witnessed an increasing number of new actors entering the field of international humanitarian response operations. The development of ICTs opened to a variety of individuals and groups unprecedented space for engagement, regardless their physical location and affiliation to traditional responders.

It was predominantly the Haiti earthquake response that pointed out the potential (and limitations) of this volunteer online engagement. Despite the fact that the actual impact, added value and benefit of these emerging initiatives is still being determined and evaluated in order to clearly identify the lessons learned and future steps, some baselines are already obvious and undeniable.

During emergency situations, the very first and the most effective responders are the affected communities themselves. However, for variety of reasons they very often remain unconnected to the traditional emergency response management systems. Simultaneously, with increased access to technology, particularly the mobile and social networks, people will not only spontaneously share information about the meal they had for lunch, but undoubtedly also information about their needs during emergency and crisis situations. Furthermore, the local and international public is seeking innovative ways of engagement in the emergency and humanitarian response.

Simply ignoring such emerging trends would be a short sighted solution – people will be sharing crisis information through any channels available, no matter if the process will be managed by someone or not. This type of information can be potentially transformed into highly valuable real time data that can significantly improve the situational awareness of non-local humanitarian responders, particularly in situations and areas where physical access to the affected community is limited. Similarly, such information sharing can significantly improve the capacity of the communities to rapidly respond to cases of emergency themselves.

The data processing and mapping based on aggregated reports from the affected population requires capacity, that can hardly be found on the ground and within traditional organizational structures during emergency. The community of online volunteers that is ready to jump-in and provide the lacking capacity could potentially improve significantly the rapidness and effectiveness of the response. The potential is obvious, as well as the challenges that need to be addressed first:

• How to practically leverage the potential of emergency information that is being shared and communicated by the affected communities and turn it into actionable data that can improve the real-time situational awareness of the local and international responders’ communities?

• How to coordinate and organize the emerging online volunteer crowd and turn it into more reliable and predictable partner for the responders’ community without harming the volunteer nature and genuine flexibility of such initiatives and efforts?

Looking for answers in the middle of emergency situations already proved to be a challenging task. It is the preparedness, continuous dialogue and open sharing that can help to gradually build-up the networks and partnerships based on trust and understanding. And this is a run on a long track. Nevertheless, the traditional international humanitarian actors already repeatedly expressed interest in understanding the functionality and potential of variety of technological tools as well as in partnering with the volunteer crisis mapping initiatives. Such collaboration can help to improve the collection and processing of data generated by the affected communities and the communication with crisis affected communities in general.

The above mentioned issues informed the decision to establish a crisis-mapping Standby Task Force (SBTF; announced at the ICCM conference in Boston in October 2010) in order to streamline online volunteer support for crisis mapping following lessons learned in Haiti, Chile and Pakistan and to provide a dedicated interface for the humanitarian community and this volunteer technical community focused on the core component crisis mapping.

In addition to that, UNOCHA (United Nation’s Office for Coordination of Humanitarian Affairs) Colombia requested the participation of the Standby Task Force in the earthquake simulation exercise in order to test the ability of the crisis mapping volunteer network to participate and contribute to the emergency operations. The request specifically mentioned deployment of an Ushahidi platform since this tool was initially tested by UNOCHA Colombia during the earthquake simulation in 2009. What else could be a better opportunity to test some of the ideas and suggested protocols that are a result of lessons learned process?

The OCHA exercise plan was to simulate an earthquake that causes significant structural damage and loss of lives. Internet is down or very spotty, making it difficult for responders to coordinate online. Mobile networks are affected, but not completely. People are still able to send SMS and make ad hoc calls, in some cases to send emails. Traditional local responders are heavily affected and their ability to respond to crisis and coordinate is very limited.

The SBTF was entering into this cooperation with specific objectives, particularly to test the SBTF activation protocols, work-flows designed for effective processing of emergency messages, to evaluate communication channels established for coordination, as well as to evaluate the ability of SBTF to respond to specific crisis mapping requests from humanitarian responders. This experience identified list of recommendations for the further development of such initiative.

While the SBTF participation in the Colombia Earthquake simulation was widely perceived as successful, the task force has identified a number of key issues that merit further attention. It is important to address these issues to improve the capacity of the SBTF to provide reliable, predictable and sustainable support to disaster-affected communities through coordination with local and international humanitarian responders.

Here is a summary of key findings:

· SBTF’s human capacity should be increased both quantitatively (the current number of volunteers should be at least doubled) and qualitatively (the teams of volunteers have to be trained in variety of fields in order to be able to provide better support).

· The SBTF internal structure should be improved with specific focus on identification and training of team coordinators.

· Deployment and communication protocols need to be further discussed and finalized in close collaboration with the response coordination actors (such as UNOCHA).

· SBTF core team should engage in ongoing discussion with humanitarian responders and other actors in the field of crisis mapping response to ensure that SBTF provides added value to the response operations and does not duplicate efforts of others.

· The SBTF should improve the ability to provide analysis support. The provision of concise analysis to responders during simulation proved very useful in increasing their situational awareness.

· The SBTF should improve the use of automated message translation system which showed very informative results that revealed its ability to expand the volunteer workforce’s capability to prioritize and categorize messages and therefore shortening the time necessary to translate and process reports.

For more detailed description and evaluation of the SBTF deployment see the attached document “Standby Taskforce participation in OCHA Colombia Earthquake Simulation 2010”.

The SBTF volunteers expressed great satisfaction with the exercise and expressed strong interest in further learning and participation in SBTF operations. Also the feedback from responders is positive and confirmed interest in future development of protocols and cooperation.

Whether we like it or, people will use any available channel of communication to share and communicate their situation during crisis. The online crisis mapping volunteer community has a unique opportunity to become a facilitator in this process that can help to turn these conversations into data that are actionable for the humanitarian responders, both local and international. And simultaneously, the volunteer crisis mapping community can respond to the demand of the traditional humanitarian responders who seek the ways to more effectively incorporate the community generated data into their standard operating procedures. The question is not whether the community generated communication is useful or not, the question is HOW to make it useful. The Colombia EQ simulation gave us some valuable ideas what next steps should be taken towards the overall objective that is more effective and accountable communication with crisis affected communities.

The Ushahidi platform is based on reports from specific places and times, so mapping items such as realtime sensors, position, or analytics poses a challenge. Consider a volunteer who is tracking oil-collecting boats in the Gulf of Mexico, or refreshing a dam gauge report during floods in Pakistan, or advising the Red Cross on how many people live in an area which reports unclean water. These are all real-world situations where volunteers have been using the wealth of information available online, but it would be more powerful to connect these visually on the Ushahidi map, alongside the crowdsourced reports.

Fortunately, both Ushahidi and CrowdMap use OpenLayers. Nearly any source can be mapped as a layer atop the reports map, with a script to create the points, tools, and interactions. Your custom points can also connect directly to the timeline.
You can save these scripts in the bookmarks section of your web browser and click to run them on any website.
Here are a couple of examples:

Drive times mapped onto Uchaguzi TZ (script from ESRI’s AgMarketFinder)

Realtime water quality and flood gauges along a river

Realtime position of boats on LA Bucket Brigade’s Oil Spill map

I have uploaded more examples including weather archives and a population estimator tool to an examples page.

To use a script, copy the link or drag the screenshot into the bookmarks section of your browser. Then, while you are on an Ushahidi or CrowdMap site, click the bookmark to activate it.
Try these examples on your own Ushahidi site, consider adaptations and additional data sources, and please share your thoughts and requests in the comments.

*Sorry for the disappearance of the earlier post, we had an outage and have recovered now.

What does the plugin do?

It provides a information qualification mechanism.

Messaging to the public submitting the information needs to include the prompt to tell people to qualify the information.

It is optional at this time, but can be customized further by future deployments. What happens is that when the information is filled in, e.g that the person either witnessed it or has provided a link to pictures/documentation, that the information appears on a grid for the administrator to consider amongst other criteria. The tool also provides an interface for the administrator to view reports within a certain proximity in order to help them make a verify/approve decision. This is done by the use of three filters. Distance, time and category.

This is an example of the Analysis tool in use on the site Uchaguzi TZ. Uchaguzi Tanzania is a joint project with Sodnet from Kenya, as well as TACCEO, which is comprised of a number of local Tanzanian organizations and led by the Legal and Human Rights Centre (LHRC) from Tanzania. The site gives you a glimpse of the citizen participation in the election. Note that this was the first time that the plugin was tested and used in the field, both ICT4Peace and Ushahidi hope to build upon the feedback received from the team working on Uchaguzi TZ.

ICT4Peace Analysis Plugin in Tanzania

The analysis plugin works best with a team of trained personnel who can ascertain the veracity of the information and indicate this appropriately on the grid. It is but one tool in their arsenal for handling information flow. With the ICT4Peace support, Ushahidi is able to provide the plugin for use by any deployer who is already using the Ushahidi platform. You may download it from our plugin site and be sure to also watch for the upcoming 2.0 release of the platform in the coming days.

Further opportunities that we welcome the greater community to help with

- Building upon this and making it mobile, imagine reporters in the field being able to flag reports and help validate information on the front end
- Further testing and feedback: If you are running a project that requires the analysis tool, try it out and let us know what you think of the tool and how we can improve it.

Combining several verification points in the software, on the front end and back end allows us to get closer to the holy grail of 100% verification. Which is nearly impossible, but a worthy goal to pursue through plugins such as this.

Update: Read more about it on the ICT4Peace website, and download the pdf.

So if we built the app from the ground up, what would it look like? How could we design the app to be as simple as possible, and yet still familiar to the user?

After some discussion about the current iPhone app prototype, I’ve realized the desired Ushahidi app functionality mirrors that of the current Apple Mail app.

So, taking those points into consideration, I created a new iPhone and iPad project using the Apple Mail app as a model. Below is a Prezi demonstrating the proposed application flow:

This is the first attempt at re-thinking the Ushahidi app, so your feedback is always welcome.

You can view the current Ushahidi iPhone code at: http://github.com/ushahidi/Ushahidi_iPhone

Note, this blog article was originally posted here.