Announcing the WikiSweeper project

Photograph from the Wikipedia 2011 Egyptian Revolution article that grew rapidly as events unfolded in Egypt

On the 25th of January this year, Egyptians took to the streets to demand the overthrow of the regime of Egyptian President Hosni Mubarak. At exactly 13:26 UTC on the 25th, Wikipedia editor, The Egyptian Liberal, created a page on .en Wikipedia titled ‘2011 Egyptian Revolution’. Following this, hundreds of edits were made to the page as events in Egypt unfolded.

When a globally-relevant news story breaks, relevant Wikipedia pages are the subject of hundreds of edits as events unfold. As each editor looks to editing and maintaining the quality and credibility of the page, they need to manually track the news cycle, each using their own spheres of reference. The decisions that are made to accept one source while rejecting others remains opaque, as are the strategies that editors develop to alert and keep track of the latest information coming in from a variety of different sources.

Today we launch an exciting new project to study how Wikipedia editors track, evaluate and verify sources on rapidly evolving pages of Wikipedia, the results of which will inform the development of Ushahidi’s Sweeper tool. The project was inspired by a meeting in late 2010 between Erik Möller of the Wikimedia Foundation and Jon Gosier, former director of Swift/Ushahidi who were struck by the common problem users of both systems are faced with when information is coming at them from hundreds or even thousands of different sources during an international news event. In particular they recognised how Sweeper could potentially be used by Wikipedia editors to collaboratively make sense of sources rather than all using independent strategies for tracking and verifying issues of importance.

In the research phase of the project, we’re trying to understand things like: How do editors evaluate sources that may be notable in one region but unknown in another? How do editors verify information in crisis scenarios like earthquakes and political unrest, or even rapidly evolving cultural phenomena when sources are situated outside the frame of reference of most established editors? What techniques do editors use to track issues around pages that they edit? What role does social media play in this process?

In the second phase, we will apply the learnings from the first phase of the project to Ushahidi’s Sweeper and RiverID tools for experimental use in Wikipedia. The research will be helpful in developing algorithms and lenses that users can apply to stream specific types of content according to different levels of authority, accuracy and trustworthiness over time.

The project is seen as a win-win for both Wikipedia and Ushahidi. The Ushahidi team hopes to integrate learnings from the project to improve the way that we manage accurate and trustworthy information. The project has been endorsed by the Wikimedia Foundation who are excited about the potential of working with Ushahidi to support Wikipedia editors.

“Wikipedia has become a go-to resource of high-quality, almost real-time information on rapidly changing events, largely because its editors constantly assess and synthesize information from a broad range of sources. Ushahidi has achieved tremendous social impact by bringing information to people in need, and we are excited to partner with them in exploring how real-time information management in Wikipedia can be made more scalable and effective,” said Dario Taraborelli about the partnership.

According to Ushahidi Executive Director, Juliana Rotich, “The Ushahidi team hopes to integrate the lessons and recommendations from the project to improve the way that we manage accurate and trustworthy information. This is important for Ushahidi deployers and users worldwide who use an array of tools like Ushahidi and Wikipedia, as they aggregate, curate and collaborate.”

The project is being jointly funded by Hivos, the Humanist Institute for Development Cooperation and the Open Society Foundation. See the WikiSweeper project page for more information or to sign up for project updates.

4 Responses to “Announcing the WikiSweeper project”

  1. Really glad to see this project has moved forward! FYI, you spelled my name wrong. It’s Gosier.

  2. Thanks, Ed! Thanks, Jon! (And sorry about the spelling error)

  3. This really gets to the heart of it: ‘they recognised how Sweeper could potentially be used by Wikipedia editors to collaboratively make sense of sources rather than all using independent strategies for tracking and verifying issues of importance.’ – will be really interesting to hear more as your research findings evolve.