Sean Conner recently asked a great question about integrating a Question and Answer service like Aardvark or Yahoo Answers into Swiftriver. Here is our approach at Team Swift…

In a Swift instance, Aardvark could be used as an additional ‘channel’ of input. Existing channels are Twitter, Email, SMS, News, RSS (any RSS feed), and Other (the catch-all for items coming in via our API). The only thing the Swift app wants to do is receive content, allow users and our algorithms to tag that content, and based on user behavior it scores the originating content source.

As an example for Aardvark: Johhny asks the question “Did an earthquake really happen in Chile?” on Vark.com on Feb 28th, only a day after the quake actually occurs. Robert responds on Vark with “No, at least I haven’t heard of one.” Vark user Jeremy responds with “Actually, Yes. An 8.8 magnitude earthquake occurred in Chile on Feb 27th.” In Swift, the answer and the accuracy of that answer is more important to us than the actual question (which just provides context).

To integrate Aardvark in Swift we’d probably write a module using their API that aggregates Answers with the corresponding Question as the ‘description’. Example of how that data would post to the Swiftriver API:

Title: “Actually, Yes. An 8.8 magnitude earthquake occurred in Chile on Feb 27th.”
Description: “Did an earthquake really happen in Chile? – Johnny”
Time: 17:08 EST
Date: Feb 30, 2010
Source: Jeremy’s user id on Vark.com
Channel: Vark API
Lat: 10.31
Lon: 01.40
Tags: 8.8, earthquake, chile

Title: “No, at least I haven’t heard of one.”
Description: “Did an earthquake really happen in Chile? – Johnny”
Time: 03:10 EST
Date: Feb 30, 2010
Source: Robert’s user id on Vark.com
Channel: Vark API
Lat: 10.31
Lon: 01.40
Tags: heard, chile, earthquake

Within Swift this is the primary information we need to verify information. Users with a careful eye will notice that we’ve included location data that Vark probably may or may not provide. We can easily extract that info from the hosted service SULSa. Here, the source is what we’re scoring. The channel is just an indicator for the user about where the content is coming from. That said, the source is not Vark itself, nor is it the user’s answer on Vark, but rather the user id on Vark.

Thus, if Robert keeps giving inaccurate answers, he maintains a very low score in Swift while Jeremy is viewed as the more trusted authority. Now this approach assumes that Vark.com offers an API that allows for this type of data aggregation which I don’t think they currently do. Perhaps, it’s a question for the Vark team?

Continuous aftershocks, office buildings scheduled for destruction, villages without access to water, gasoline shortages, and looting. These are some of the 800+ incidents my fellow students and I mapped during our first week in the situation room at Columbia University’s School of International and Public Affairs (SIPA). The death toll from the February 27 earthquake is much lower than the number of lives lost since January 12 in Haiti, but the 8.8-magnitude quake in southern Chile was one of the largest on record, and coordinating information, needs and responses on the ground is essential.

Near real-time crisis mapping

An initial training session at Columbia University the Monday after the earthquake drew 60 students—during SIPA’s midterm examination period. A core team of six students is working to ensure that the growing list of volunteer names translates into more hours spent monitoring and mapping. Other schools within Columbia University are starting to get involved, and the team has reached out to contacts at New York University. One of our core members took a quick break from the Situation Room to head to United Nations headquarters after individuals from the United Nations Development Program (UNDP) asked for training on the Ushahidi-Chile application. Most importantly, as students continue to monitor traditional and social media, more reports are reaching the Ushahidi platform from on the ground in Chile.

The core SIPA Team

Some students were initially skeptical of Ushahidi-Chile. The Chilean government had said international aid was not necessary, and the value of mapping information from news sources and Twitter—in short, information that can be found elsewhere—was questioned. However, by Monday, messages communicating the needs and locations of people who lacked basic supplies and water were coming in. It seemed that our team had uploaded enough information to be truly useful on the ground. Soon, Chilean organizations including Red Salvavidas and Chile Ayuda were contributing reports daily.

As a journalist, the flow of information within Ushahidi has been immensely interesting to me. The process that our teams of monitors, mappers and administrators go through to produce the Ushahidi page is not unlike the path a journalist takes to report and produce a story. In both cases, the reporter seeks information from overlooked sources and tries to verify the data. Small bits of information combine to produce a larger picture, giving an overview of a particular situation. In Ushahidi, the picture that emerges is an interactive map, rather than a narrated story.

Crisis Mapping volunteers at SIPA

The SIPA team has touched base with technology students from Talca, in one of the hardest regions hit in Chile. The Chilean students now have administrative abilities on the Ushahidi-Chile page. The goal of the SIPA students working with Chile-Ushahidi is to transfer the platform to an organization in Chile by the end of March, but to leave behind a trained group of crisis mappers at SIPA who will be ready to assemble and share information whenever and wherever the next disaster strikes.

The diagrams below illustrate how this dataflow works. Text passing through SiLCC are parsed, tags are extracted, those tags are then reapplied in the Swift River UI. There, Swift attempts to build relationships between tags. (ex. items tagged with “chile” and “earthquake” are likely related. However items tagged ‘chili’ and ‘earthquake’ likely are not.) Of course other factors are considered like date, time, the point of origin and location of the content creator.

One of the services running within SiLCC is another service called SLISa, which we like to call Lisa (because the ’s’ is silent, hehe). SLISa is the Swift Language Improvement Service App and it trains SiLCC to learn from user interaction. When users of Swift edit or flag tags as inaccurate, SLISa is the service that creates all the conditions that helps SiLCC to learn from it’s mistakes and improve for the future.

SiLCC is an open source project being developed in Python using the pyNLP toolkit. There’s several additional layers of text parsing that I haven’t touched upon including how SiLCC deals with SMS txtspk and Twitter picoformats like hashtags but more on that in a future post!

More on SiLCC at http://swift.ushahidi.com/extend/silcc/. If you have a passion for machine learning, large data sets, and intricate algorithms you might also consider joining the Swift River Google Group or our public Skype Chat.

The Alpha release of Swift River, Version 0.0.9 Rumba will be available to the public on March 31, 2010. Developers can find always find the latest working build and issue tracker at http://github.com/ushahidi/Swiftriver.

Working in the tech sphere, it’s the power and passion that people have that never ceases to astound me. Following the Jan. 12 earthquake in Haiti, people around the world contributed an incredible outpouring of support for the people of Haiti. Now, as that support expands to Chile, it is evident that we are participating in a game changing moment.

On January 12th when the earthquake in Haiti struck, Digital Democracy had two team members on the ground looking at the economic livlihoods of young Haitians . Worried for their safety and the greater loss of life, my colleague Emily and I immediately joined Patrick in the Situation Room at Fletcher, and collaborated with the core team to establish a system that allowed hundreds of volunteers to glean emergency instances from the ground and place them on the map for response. Those first volunteers have trained many more, and their tireless work has directly saved lives.

When an 8.8 magnitude earthquake hit Chile on Saturday, we were called on again to train volunteers. The Fletcher Ushahidi team, their hands full with Haiti, was able to rapidly set up the site for Chile.Ushahidi.com, but needed help training new volunteers. Digital Democracy collaborated to adapt the training module for the Situation Room at the Fletcher School at Tufts University, and on Sunday night we conducted an hour+ training on Skype for a core team at SIPA at Columbia in New York City who are now running the Chile operations. Their ability to quickly respond to the devastating emergency in Chile, despite it being midterms week, speaks to the dedication and passion of the volunteers. The 10 people we trained on Sunday trained 40 more the next day at SIPA, plus a growing number of volunteers in Santiago.

For us, this has been a particularly rewarding process, as we’ve been following and working with Ushahidi since fall 2008. Watching Ushahidi evolve from mapping post-election violence in Kenya to elections reporting when we helped with VoteReportIndia, we’ve worked to harness Ushahidi to empower ordinary citizens. At Digital Democracy, we’ve applied Ushahidi to map human rights abuses in Burma through Handheld Human Rights and peace mapping in Kenya through Sisi Ni Amani – “We are Peace.” In each case, we’ve worked with local community organizations to determine needs and apply the technology – Ushahidi – to best meet their needs.

Our mission is to utilize technology for civic engagement, and these examples demonstrate how technology can encourage deeper engagement with the world around us. At the core, it’s not the technology but the people who use it. Ushahidi is a tool that’s a key part of our arsenal because of the open community around it. As technology enables more and more people to contribute to meaningful actions to save lives, I for one am excited to see where it leads.

One of the ideas that hit me was the need for a toolbox of technical tools that could be used by election monitoring groups and citizens both before, during and after the elections.

Understanding the Framework

Most of us think of an election as an event, I did too. Koki Muli provided us a with a great framework to understand the election process as a whole, using this visualization for everyone to see that it is indeed a long-term process, not an event.

The Election Cycle

Throughout the days of talk, a lot of information was given about elections, and specifically election monitoring as a practice and how it actually goes down in the real world. I’m not an expert in election monitoring, so there was a lot of education for me, but putting on my technology hat I (and the other tech guys present) were able to come up with some ideas for tactical-level solutions that might be useful to these election monitoring groups.

Starting with the Issues

I don’t have time to run through all of the issues, but here are a couple that have a technology component that could be created to help with them.

Legal Framework
The legal framework for elections starts many years before an election happens, but might be the most important element in an election (in a law-abiding state). These are high-brow activities, where law professors and legal experts write in a format that isn’t easily understood by ordinary people.

The question then is, how could this legalese be translated into a format that is digestable by normal citizens, and how could they then give feedback on what they think of these new laws? I’m already thinking of a tool that can be built for this, having a lot in common with (oddly enough) digital Bibles and how they allow multiple levels of commentary by readers and scholars.

Election Monitors
Registration of election monitors is not rocket science. Many times, the election monitoring groups get funds at the last minute and they go out and recruit anyone above a certain education level within the different constituencies. There is little vetting, some training, and no understanding of their values and ethics for when election day actually arrives. It’s no surprise then that when election day comes, many of these election monitors turn out to be little more than party infiltrators, sometimes abusing the system worse than normal citizens.

What tool can we build to support creation of a database of known election monitors, with full information about them, so that when irregularities or criminal activities are done by or with them that it can be tracked. Having a historical database of election monitors will allow for pattern mapping and even a blacklist for the civil society groups when they go out to find monitors for the next elections.

Real-time Data Collection
During the campaigning and the election day itself, it’s possible to collect information on irregularities from both trusted/known sources (including media, government and civil society), but also from ordinary citizens in the Ushahidi-style of crowdsourcing from citizenry. This includes media monitoring as well as gathering of historical and demographic data so that real-time analysis can happen.

Tools like Ushahidi and Managing News can be utilized here, this is what they are built for. The different types of news sources are complimentary, allowing layering of new, real-time data on top of demographic and historical election data.

Onto the Toolbox

These issues and challenges throughout the election cycle cannot be set up and deployed in the last few weeks or months if they are to be effective. We all need to sit down now, come up with the tactical and strategic-level tools that are needed and deploy them within the organizations who can best utilize them in the election process.

• What tools can be built?
• What tools exist already?
• How can we create a toolbox that election monitoring groups, and citizens, can choose tech tools from and deploy?
• Has this already been started somewhere else?

Add your thoughts and ideas in the comments below.

Let’s see if we can come up with some ideas, as technologists especially, of tools that can work in conjunction with one another to strengthen the election monitoring and the election cycle as a whole.

Reinier Battenberg had a good slideshow with examples of where tools are needed at different parts of the election cycle. I’m including that here as well as food for discussion. (click the image to download his Powerpoint presentation)

10 applications for use in election monitoring

Finally, here’s a list of election monitoring-specific tools we could think of, what would you add to it?

• Ushahidi
• FrontlineSMS
• Managing News
• OpenStreetMap
• SlingshotSMS
• RapidSMS
• SMS Reception Center
• Budget Tracking (app by SODNET)
• International SMS gateways (Clickatell, Intellisms, etc)
• Swift River
• SMS Turks
• Pentaho
• MySociety.org
• NDI’s election monitoring tool (ASP based)
• CiviCRM
• WordPress
• CMS tools (Drupal, Joomla, etc.)
• Freedom Fone

The elections in Togo are being held today, and Kokou Etou set up this implementation to help aggregate information. He partnered with IFES (international Foundation for Electoral Systems) and other organizations working in Togo to visualize the info and put it online.

For some background on the Togo Elections, read this piece by Amourlaye Touréand Paul Chick of IFES/Togo.

There are three ways to submit information
1. Text your location and info to 0012673935789
2. Email info@togoelection2010.com
3. Submit reports on the site

Do support this initiative and visit their site here, please do let others know within Togo to check the site for information. This will be particularly helpful to Kokou and his partners as they have already done some preliminary outreach through online forums and radio ads.

And of course thanks to The Fletcher Team for holding the fort during the first 48 hours. Last but certainly not least, thanks to the Ushahidi Tech Team for once again rallying to the cause after their massive effort on Haiti. Our colleague Oscar Salazar from “Ushahidi-Mexico” was also pivotal in getting the Ushahidi-Chile platform off the ground and will continue to be. Clearly, this is about building a community, a volunteer response community, more than anything else.

This is why I’m excited that another School of International Affairs has set up a Situation Room and taken the lead on an Ushahidi deployment. The Fletcher, DC, Geneva, London and Portland Teams already demonstrated the incredible multiplier effect that is possible thanks to their lead on the Ushahidi-Haiti project. There are now three core Ushahidi Situation Rooms in the world, Boston, New York and Geneva. They are fully trained and continue to train others. So if you are based at a university and want to set up your own Situation Room, then please feel free to contact me: patrick@ushahidi.com.

The earthquake in Chile and resulting tsunamis have not caused the widespread loss of life that many initially feared just hours after the 8.8 magnitude earthquake. But the advantage of platforms like Ushahidi’s is that they can be deployed regardless, just in case.  Consider this the Precautionary Principle of Disaster Management. And thanks to a growing community of CrisisMappers, these precautionary deployments can happen more often and faster.

Stay tuned for guest blog posts from the Digital Democracy and SIPA Teams.

Customizing Ushahidi-Chile

We all realize that this is a risk coming so soon after the massive efforts around the Haiti quake. But we didn’t have all the answers then either. I will collaborate closely with the Fletcher/Tufts team and look for volunteer groups with universities in Mexico and Colombia, for example. I’ve already reached out to my good friend Oscar Salazar who launched the successful Ushahidi-Mexico platform and he is mobilizing his network of volunteers in Mexico. I’m also reaching out to our South American colleagues at GlobalVoices.

We’ve already set up a long code that individual can use to text in their location and urgent needs: +44 762.480.2524. Note that we are working with our partners to set up a short code for Chile. So the one listed here 4636 is not operational. Please standby for more information on a local short code. In the meantime, you can submit reports directly online here. You can also send an email to chile@ushahidi.com with specific location information and urgent needs.

We will provide further updates on this blog and also on our Twitter feed @ushahidi.

]]> http://blog.ushahidi.com/index.php/2010/02/27/volunteers-respond-with-ushahidi-chile/feed/ 28 http://blog.ushahidi.com/index.php/2010/02/23/ushahidi-haiti-closing-the-feedback-loop/ http://blog.ushahidi.com/index.php/2010/02/23/ushahidi-haiti-closing-the-feedback-loop/#comments Tue, 23 Feb 2010 16:24:17 +0000 patrick http://blog.ushahidi.com/?p=1566