This week several members of our team attended the 3rd International Conference of Crisis Mappers (ICCM) that was held in Geneva, Switzerland and we were blown away by the turn out (Follow #ICCM on Twitter). This community has grown substantially over the last 3 years – when we attended the first conference held in Cleveland, Ohio back in 2009 there were about 100 attendees. Now there are more than 400 gathered here discussing crisis mapping and the challenges they face. There are really 3 main things that we keep hearing as it relates to GIS and mapping:

• How to collect and organize data from the crowd around the globe as it relates to crisis
• How to enable action from this data
• How to build, collect, and maintain reference data for crisis

Collecting and Organizing Data from the Crowd

As for the first topic, collecting and organizing data from the crowd, there is a good tool developed by our partner Ushahidi. Ushahidi is a platform that takes crowd sourced information like SMS messages, categorizes them, and displays them on the map as a point or cluster of points showing relevant location. The result of this effort is a point map that begins to illustrate where individuals are vocalizing need or disseminating information.

Enabling Action

While points are a good start, we have been working with Ushahidi and a focus group comprised of both GIS and Crisis Mappers to define requirements for new tools that will support analysis of this data. We have heard the need for spatial and temporal analysis tools within the Ushahidi platform as well as the need for tools to bring Ushahidi data into ArcGIS where analyses can be run.

We are very excited to announce that we have made the first tool available to support these requirements-the ArcGIS add-in for Ushahidi which can be downloaded and quickly added to ArcMap. This add-in allows you to connect to a Ushahidi instance with the API enabled, convert text between languages, and download the Ushahidi data into a geodatabase. This opens up the feed of data being captured in Ushahidi to the rich spatial and temporal analysis tools within ArcGIS allowing users to empower action and inform decision makers using a sound scientific approach. Curious how this might work? At the bottom of this blog is a good write-up of using this tool to analyze information coming in from the current Thailand flooding.

Reference Data

Having this information and resulting analysis is great, but without good reference/basemap data as a backdrop, it is hard to understand the context of any situation. A common theme discussed here in Geneva is the isolated and rural nature of many locations and the lack of base data. While there may be global vector and raster datasets available, they are often either severely dated or at a scale that does not support local action.

One data set that supports crowdsourcing of basemap data is OpenStreetMap (OSM). OSM is constructed and edited for all types of use by a global user community, and yes you can join! Crowdsourced basemap data is particularly valuable during a crisis response where current and often high resolution data needs are compounded.

In support of this open source effort, Esri offers a desktop tool for ArcGIS that allows you to join the crowd by editing and publishing to OSM from within ArcMap. This tool also allows you to download OSM data over your area of interest directly into ArcGIS for use in a disconnected and offline environment common in response. You can download the ArcGIS Editor for OSM.

Esri is committed to supporting the crisis response community thru our Esri Disaster Response Program as well as thru the continued evolution of ArcGIS to support the collection, management, analysis, and visualization of data in a collaborative environment. While these tools don’t solve all of the problems we face as crisis mappers, we hope you will find them valuable in your arsenal and as a starting point to a dialogue around enhancements to and construction of additional tools or data. We are already looking forward to ICCM 2012 in Washington, DC!

ArcGIS Add-in for Ushahidi

Getting Started

To get started, first download the Esri Add-in for Ushahidi. Once you’ve installed the Add-in you’re ready to add it to ArcMap. Go to the Customize menu | Toolbars | Customize… located at the very bottom of the list.

In the Customize dialog, click the Commands tab| type Ushahidi in the Show commands containing: text box. You will see the Ushahidi2ArcGIS Add-In listed in the Commands pane.

Drag the Ushahidi2ArcGIS command to any existing toolbar in your ArcMap window. You must drag this command onto an existing toolbar, not just onto the map. You will see a new button with a blue circle icon show up on your toolbar.

Connecting to a Ushahidi Instance

To connect to an Ushahidi instance click on the Download Ushahidi Reports to ArcGIS button and enter the parameters.

The key parameter to enter is the API Endpoint. This is the main URL to the Ushahidi instance, such as Thailand Flood Crisis Information Map or Bushfire Connect. You also have the option to translate the data (using Bing Translator) for either just the categories or all the incident data. Note that the more you have to translate and the more records you have the longer the time it will take to download and create the data.

Once the data is downloaded it is added to a layer. The tool automatically sets the data to render on Unique Values in the Category Title field and uses the default colors from the Ushahidi instance.

Additionally the HTML Popup set and the description of the layer has been populated to reflect the source, date and time the data was downloaded. This is critical temporal information for Crisis Mappers.

Perform Further Analysis

Now that the Ushahidi data is downloaded you’re ready to perform further analysis. You can look at data over time, limit your focus on specific categories, or do more in depth spatial analysis.

For example if we wanted to get a better understanding of the reports of flooding around Bangkok we can focus on those reports (i.e. just one category of the incidents).

We can then visualize the data as a “heat-map” (by using the Kernel Density tool).

To get more detailed information on the data and to further determine significance of the reports we can use spatial statistics to determine if there are non-random spatial patterns and statistically significant hot or cold spots in the data. To do this we can use the Spatial Autocorrelation and Hot-Spot Analysis tools. The basic workflow is shown here in ModelBuilder:

The results of this analysis are highlighted in blue on the map as areas (1 km^2 ) that have statistically significant clusters of flood reports (in this case 8 to 27 per km2 ). These are the high priority areas that should be focused on first. Then we focus on questions like why are there so many reports in this area? Is there a single authoritative source of SMS feeds on the ground or is there a large group of people that need help?

This is just one type of analysis that can be run in ArcGIS Desktop from Ushahidi data. We are interested in learning more from you in the type of analysis that you’ve run using this tool. Please use the comments section of the tool and let us know how the tool works and what needs to be improved.

*****

Thanks to the ESRI team for creating this valuable plug-in.

During the 2011 Liberian Election process, Ushahidi Liberia proved to be an invaluable team member for the United Nations Mission in Liberia (UNMIL).  Their crowdseeding efforts provided the people of Liberia, UNMIL and others, with timely access to objective reports from around the country.  Lighter and more agile than the UN structure, the Ushahidi Liberia team was able to collate nearly 5,000 reports from perspectives previously not readily accessible to most observers.  Additionally, the constant communication by phone, email and in person between Ushahidi Liberia and the UNMIL Joint Elections Operations Center (JEOC) personnel allowed for cross fertilization and information vetting, improving the fidelity of reporting for all consumers.

Situational Awareness Tools

The best sensors are the human senses. Broadly leveraging what these sensors acquire is impossible without standards, tools, training and leadership, structure that is both formal and informal.  The affected, on-the-ground responders, and providers with reachback capabilities create a circle of dependency that is often broken because of the lack of structure. On the flipside the ability to achieve and maintain situational awareness was and remains bound by the lowest common denominators of an organization and its personnel. The Ushahidi platform allowed UNMIL to break through some of the challenges of:

• Knowing what information is important, available and where to find and leverage it
• The End user’s
  • Education level
  • Language skills
  • Computer skills
  • Motivation level
  • Access to tools (power, computer, internet, phone)
  • Training on the tools

Information flows during Liberia's 2011 election

In the end, success was based on preparation and relationships.  The Ushahidi Liberia team provided access to resources and information that UNMIL simply could not have leveraged in their absence.  Constant communication by phone, email and in person between Ushahidi and the UNMIL JEOC allowed for cross-fertilization and information vetting.

About Ushahidi Liberia

1. Ushahidi Liberia has direct partnerships with 16 different NGOs (international and local), civil society coalitions and the government. Among these partnerships there are many other indirect partners (example:  Elections Coordinating Committee is a partner, but they are composed of 30 organizations; IFES has 20 CSOs that Ushahidi has trained who are out in the field reporting to them, etc). Ushahidi has also provided a map for UN OCHA made at their request.

2. Ushahidi had about 7 volunteers during the first run-off.

3. Total reports on elections instance since January = 4,954 (that are public)

4. Androids – Ushahidi had 4 of them running the free election shortcode and also the free long number for a national early warning map (LERN).

5. Ushahidi Liberia’s VSAT connection during the election was 1054/512 kbps (the fastest public internet connection in Liberia), now reduced (due to high costs) to 768/256 kbps – it is a dedicated C-band connection available to Ushahidi Liberia users in their facility. They have 16 computers running open source software. Their Dir. of IT, Dir. of Training, and Program Director are based in-country, with a Tech Lead based in the US.

Data from USAVE to end users

The geography of crisis and response

Geographic location, type of crisis, responder specialty and organization greatly impact the way in which the individual will operate.  However, each shares common, basic geographic (map) data requirements.  Imagery, terrain, political boundaries, infrastructure and hydrography are the minimal data sets required for any type of fieldwork.  Depending on the event, political, social, demographic, medical, refugee, reported violence and other kinds of information may become most critical to obtain.  For the purpose of this thought process we will focus on the base geographic data requirements.

LOCATE: Where am I?  Where is the disaster?  Where are those in need?  Where are response resources?  How do I get to the resources?  How do I get the resources to the affected?

- Country, city, town, base camp and devastated area

- Affected, other responders, and external partners

- Infrastructure (water, power, communications, sewer, medical, transportation, and security)

- Resources (water, food, shelter, medical, transportation, communication, security)

COMMUNICATE: Information, requirements, coordinates, coordination, challenges and successes.

- Affected, other responders, higher headquarters and external partners

- Collected field data, open source

- Data, space and ground based sensor data

- Needs, challenges and successes

FACILITATE: Response, assessments, support, capacity building, documentation & retrograde.

- Information collection and sharing

- Resource acquisition, delivery and employment

- Initial and sustainment training

- Documentation, configuration control

- Responsible turn-over to and departure from Host Nation

Just on the tip of what is possible

Looking to the future

If space-based providers can push data down to the lowest common denominator in a timely manner, in a format they may leverage, the future is bright.  If not, expensive space-based products will remain tools employed by the elite and an educated few “in the know”, remaining invisible to those it would best serve.  Using geographic information systems is no longer a luxury, it is a necessity across the spectrum of United Nations mission sets.  Manual procedures of reporting, filing and analyzing information should be placed behind us.

The success of the next crisis response begins today.  With the right equipment for the mission, end users may even operate disconnected from the grid, know where they are, collect and share information with others on the ground and, when finally connected, receive and transmit vital information to all interested parties. Each scenario requires the end user to pack appropriately based on factors such as financial resources, logistics restrictions, availability of infrastructure within the impact area, on-ground transportation and individual capabilities.

To give the end user access to harnessed capabilities one may consider providing equipment and training so the value may be broadly shared amongst operators instead of unintended hoarding amongst technical specialists.  If the end user connects to the grid, they will be able to receive timely ground and space based data like high resolution post-event imagery from numerous sources, as well as interface with the “cloud.”

The future is already here

A circle of dependency has become apparent between organizations, formal and informal, and the crowd.  They are intertwined, even though some resist.  The bounds have become tighter and the value greater among those parts of the circle working with, rather than against, each other.  During the election season, Ushahidi Liberia provided an environment of professional cooperation necessary for the cultivation of numerous complex relationships.  Together, we have taken a very large step forward into the future. Although, likely never to be the same, the road has now been traveled.  There is no going back.  Know the past, anticipate the future, show the way!

Run-off election results on the map

In the days between the run-off and NEC’s first announcement, the Liberia Media Center published unofficial rolling results from field journalists covering the polling places. These initial results were the first and only vote counts available to the Liberian public, and were published right away on the elections instance as “LMC run-off vote counts” (just below categories, under “other layers”). The Elections Coordinating Committee, active observers during the first round, sent out another 1,750+ monitors on run-off day who have now returned to iLab and are diligently recording the events that occurred at Liberia’s polling places. These ECC reports from the first round can be found under the category “ECC election day monitoring”). Second round ECC reports will appear on the map once the data operators have received all polling checklists.

ECC polling station reports from the first round

During the last week, the majority of the map’s reports have been about the events surrounding Monday’s CDC rally turned violent, and minor incidents during the run-off and the following day. Some of the incidents reported include: attempted ballot box stealing, the torn down polling station results, tear gas fired by police when a crowd refused to give up ballot boxes, the closure of three major radio stations on charges of hate speech, the burning of a major radio station that is still under investigation, and Thursday’s release of 84 persons arrested during the rally.

Picture from the map of an opposition rally turned violent

Young voters proud to cast their ballots

Since Wednesday, the Ushahidi Liberia team has mostly been mapping news from the local media and international observers such as the Carter Center and ECOWAS. We plan to continue updating these vote counts until the final results are released. However, with recent news that opposition leader Winston Tubman will contest the results and seek legal action to hold another election next month, this election map may not go quiet simply because the last vote is counted.

UP headquarters in Paynesville burned

In the last week, the Ushahidi elections instance has also been updated with new features.  As of last Friday, the instance has been upgraded with latest version of the Ushahidi platform, version 2.1 (Tunis). One of the most exciting features of 2.1 is the Powerful Reports Filter that allows users to quickly sort through reports according to certain dates and categories, verification, media, location and more.

The new Density Map Plugin

Another new feature on the instance is Ushahidi’s first Density Map plugin by John Etherton. Located on the right side of the homepage, just above the categories, the Density Map option makes it easier to separate reports by geographic region – in this case, by county. All reports that have been associated with a particular county will show up using the Density Map. This feature was requested by UN OCHA as well as other local partners, and serves as a small step towards making the Ushahidi platform a data analysis tool.

ECC data operators at iLab

The Elections Coordinating Committee, based at iLab Liberia during the last week, has been compiling detailed reports from monitors at each of the country’s nearly 4,500 polling places; these are being uploaded to the elections instance and can currently be found under the category, “ECC polling place reports”.  The National Elections Commission (NEC) has released preliminary results during the last week that can now be found as layers on the elections instance (go to “other layers” section under the categories listing).

The Ushahidi Liberia team was curious about who was looking at the elections instance, how they got there, and where in the world they were sitting when they looked at it.  After studying our instance’s Google Analytics, we found that in the last week the map has received 3,533 unique visitors that spend an average of 3 ½ minutes perusing its contents. Interestingly, more than half of this traffic is routed via the Liberian Observer, a national newspaper popular on the ground and online. The majority of the traffic came from the United States (not surprising considering low Internet penetration in Liberia), but what peaked our interest was where in the States: Minnesota, Pennsylvania, Maryland, New York, Georgia – states with the large Liberian Diaspora communities.

Elections instance viewers in the US (graphic via Google Analytics)

This was our team’s first indication of the Diaspora’s engagement with the electoral process from abroad (not to say there haven’t been many), and it widened our perspective on the instance’s audience. Perhaps, in a country without much local Internet access, it cannot be simply stated that a tool like Ushahidi is irrelevant. In today’s world, a country’s borders extend far beyond political boundaries, and interactive mapping tools such as the Ushahidi platform start to reveal the interconnected webs that criss-cross our globe.

This evening, the NEC announced that 99.9% of the presidential votes have been collected; with no clear winner, Liberia faces a run-off election on November 8^th.  The first round saw an impressive voter turnout rate of more than 70% (of registered voters), making many hopeful that Liberians will come out in full-force next month. The Ushahidi elections instance will continue to track the electoral process for Liberians at home and abroad, and for all of us who care deeply about the outcome of this country’s first self-run democratic election.

Liberians waiting to vote on election day

The Criteria: Recognize deployments, deployers, teams of deployers, individuals and communities and feature Ushahidi and Crowdmaps that do one or more of the following: build community/teamwork, demonstrate outreach, showcase/expand the open source software (new themes or code), have a unique or timely topic, be active and more.

Deployments of the Week

(Globe-sized Graffiti in Montreal, QC. Photo by Heather Leson)

• October 7 – 15.10.2011: United for #globalchange October 15 (global)
• September 30 – Mapping community needs in Chisinau (Moldovia)
• September 23 – Mapping community needs in Livorno (Italy)
• September 15 – Election Monitoring in Guatemala and Zambia
• September 9 -Hospitals and Medical issues (Barcelona, Spain)
• September 2 – Crisis Response – Irene Hurricane Recovery with an honourable mention to VTIrene Crisis Map (USA)
• August 26 – Reporting Corruption/Violence (Bulgaria)
• August 19 – Corruption, Traffic – Lagos (Nigeria)
• August 12 – Agriculture – Agrotestigo Plant Disease tracking (Argentina)
• August 3 – Civil Society/Community Building – Fix your Street (Ireland)
• July 26 – Zabatak – Corruption (Egypt)
• July 18 – Available Petrol (South Africa)

How you can help:

We would like your input on Deployments of the Week. These deployments should be active and fit some of the criteria noted above. Please be sure to mention your deployment to us: tweet @ushahidi or @crowdmap or post to our Facebook page. Some deployments are not publicly available as deployers can opt out of search. The Ushahidi team has been reviewing and voting on the deployments on a weekly basis. We will look into opening up the voting.

Deployment Types

We’ve found that there are a few types of deployments, but are watching as it expands:

• Election monitoring
• Crisis and Emergency Response
• Civil and Community Building
• Activism
• Storytellers, Business and other topics

Who are the user groups?

Our community is so diverse in skills and general types of user groups. We talked previously about the types of community members by skill, but these are some of the core groups who deploy:

• Media
• Civil Society organizations
• Deployment Partners
• Non-governmental/Governmental associations, UN (local, national, global)
• Community-Based Organizations (CBOs)
• Citizen Media
• Volunteer teams like the StandbyTaskForce, CrisisCommons, Humanity Road, Geeks Without Bounds and more
• Researchers/Academics
• Technical developers
• Mappers/GIS
• Remarkable Individuals

and more.

People are telling map stories of corruption, urban design, neighbourhoods, city-building, cooperation, traffic, preparedness and more. Giving a visual life to our communities and their issues, strategy and needs, here are some Ushahidi and Crowdmap examples:

Fix My Street (Dublin, Ireland)

South Dublin County Council is taking reports about road conditions, litter and drainage:

Lagos Traffic (Lagos, Nigeria)

Lagos (city and state) is the second fastest growing city in Africa. The Lagos Traffic Map hopes to account for the appalling situation that inhabitant and motorists go through everyday; helps to give out early warnings to air, water and road users; and helps emergency managers in coordinating and managing mishaps. It is a two-month pilot project lead by Nigerian volunteers. Follow the hashtag #lagostraffic.

By the City/For the City (New York City, USA):
By the City / For the City is a project created and executed by the Institute for Urban Design. They worked with the Project for Public Spaces (PPS). digital team to develop the platform using their Ushahidi-based Placemap tool. Using a map to build open government and civic engagement, PPS focuses on digital placemaking by using “integration of social media into Placemaking practices, which are community-centered, encouraging public participation, collaboration, and transparency.” While this year’s completion is complete, the Institute for Urban Design is planning next year’s map.

Segnala Livorno (Livorno, Italy)
They encourage the community to map their problems every day from potholes in the streets, the beaches are dirty to degradation in the parks.

Mapping social problems (Chisinau, Moldova)

Community Planning and Design: Connecting professionals with communities (Toronto, Canada)

What is your city story?

Other City building examples:

• Trentino (Italy) -local newspaper
• Overlap in Nairobi, Kenya (area) (tracking reckless driving and traffic issues)

Heather L.

• Ushahidi needs stronger, unified documentation (a master wiki)
• Provide an accountable customer service work-flow
• Create an open development process, and
• Increase community programming to support mentorship and training.

Who is our community?

Ushahidi is used by five primary groups: international/local development, crisismappers/community capacity/civil society, media/communications, non-governmental/non-profit and governmental organizations. Types of community members can be researchers, academics, policy analysts, software developers, system administrators, logistics, event planners, deployment partners, activists, translators, coordinators, project managers, technical writers, translators, graphic designers, community outreach, videographer, volunteer coordinators, curious and everything! Whew! What an amazing swath of knowledge deployments convene. We are in awe.

What can we improve on?

Ushahidi needs support:

You told us that our support process is confusing and, at times, hard to reach. Support questions seem to be directed to the Developer Skype Chat, Forums, Wiki, mailing lists, individual team members, and etc. So, we are investigating streamlining our communications for technical support channels. We use Tender for all the general contact questions. We use Github, Forums (Vanilla) and Redmine for technical content. We’ve put together a list of requirements for the best fit solution to this. If you have any recommendations on the best tools to use or combination of tools, we’d welcome the input.

Create standards and protocols for releasing news about serious bugs and security risks over email. I don’t have the time to constantly monitor the Skype chat.

Be more responsive, and improve the quality of the product. Meet the needs of the community…

Ushahidi documentation is unclear. Open up the development process.

Make full use of wiki tools, I think we need to construct clear taxonomies to know what it is that is being categorized.

Emmanuel Kala and I have been tasked to determine the next steps with the wiki. Stay tuned on this journey. We’ve been reviewing how our friends in other open source communities keep a healthy, happening wiki alive and relevant. If you would like to help us on the wiki project, please just drop us a line.

…monthly conference calls where developers talk about what they’re working on and members of the community can share their thoughts on what’s working and what isn’t.

Increased programming to support mentorship and training.

I’ll +1 the idea for more videos, more webinars, and more visuals of even simple administration tasks. Ushahidi’s translation community will probably never be able to keep up with the platform/documentation changes in all of the required languages. The visual instruction, even if the verbal instruction is not understood, provides some understand of basic to more advanced functions.

Caleb Bell and I are working on a video channel for our training. We completely agree that video + language is so important to aid in supporting the ever-growing global, multi-lingual community. We will also be holding events and webinars for the various user groups to support the diverse deployment communities.

Also, two people requested More Cowbell. We’ll incorporate that into the next community call to be scheduled for early October.

Thanks to the 55 people who responded to the first community survey and to the countless people who talked with me both online and in-person. We are tasked with addressing these critical pieces to support your work. And, as we formulate the approach, we’ll share and ask for your input to test or improve it.

Chris’s presentation at the Silicon Prairie News Start-up Demo Night

Some of the best practices of the MightyMo team included their detailed data workflow using map overlays, granular report forms combining both traditional governmental and social media reports, and multiple verification methods. In addition, twice daily they curated a MightyMo River Gazette page using the free tool paper.li. This page enabled them to provide headline news. Framing and analyzing the map content is a critical deployment best practice for outreach. The MightyMo “About” page should be a standard template for Crowdmap deployers as it clearly explains the project mission, contact information, toolset descriptions and a basic project plan. The clear Frequently Asked Questions (FAQs) could also be modified for other deployments as they explain the project plan/workflow, including answering the essential question: “how to help”.

To learn more on architecture of the MightyMo project and their best practices, see Chris’s Knowledge Discovery / Modeling & Simulation Workshop MightyMoRiver presentation (PDF). And, learn about the project origins.

The Ushahidi team is always happy to improve the deployer maperience. Also, it is gift to have people like Chris mentor and share knowledge with fellow community members.

Thanks!

We are honoured to have our local community and some guests like UNHCR and NetHope join us to talk about best practices and improvements for deployments and Ushahidi.

Here is our Ustream for the day: (recorded)

Streaming live video by Ustream

Our live blog:

Back to it,

Heather L.

Building Bridges

Rukia and Tobias provided further feedback on their deployment: successes, observations and things to improve.

Highlights:

• OUTREACH: Communication targeting of audiences differed for rural and urban areas and diverse age groups. For example, rural areas were reached by radio rather than newspaper. However, the radio programs were sometimes aired in different vernacular language to accommodate all listeners. The impact was great, even neighbouring countries such as Uganda received the information from the website and created their profile online. This required more intensive report verification based on location. The power of mobile in .ke is amazing. Telco providers like Safaricom will allow short codes to accompany hash-tags which will redirect to a website, like Building Bridges. The campaign was successful due to the SMS program and pervasive use in .ke.
• REPORTS: As part of the campaign, the Building Bridges team telephoned over 6000 people. These calls were often to verify the content or to request clarification. Some content was translated into English for the map from telephone calls. Some reports were web-based submissions in Swahili. It would have been useful to have a bilingual map: Swahili and English. People who filed reports often required feedback or follow-up on their content. These involved counselor-type telephone calls. This community approach was essential to build comfort and trust. Some report providers preferred that their content be provided verbally and their report listed by “peacemaker” to maintain their privacy. The majority of the reports were submitted via SMS with the other reports far below in numbers (in descending order): telephone calls, email and, lastly, web-based form reports.
• KEY LESSONS: The plan was revised to accommodate more SMS reports (sometimes 700 a day). Education (digital literacy) was a large component of the telephone calls for feedback loops. The number targets were given less weight over time in lieu of qualitative content and relationship-building with the reporters. And, this type of project is easily replicable in other regions using Ushahidi’s platform.

A very special thanks to Rukia and Tobias for their contribution to this project and for their time today. Check out their other projects including mobile education games such as Get H20.

There are still a few spots left open for the Ushahidi .Ke Evaluation Launch. Register via Meet-up. We will be testing toolboxes, talking best practices and building forward.

Heather L.