A Course on Collective Intelligence

A final-year and masters level computer science course about the computational aspects of collective intelligence and the wisdom of crowds.


To many the promise of artificial intelligence (AI) — the creation of a computational intelligence to rival our own — represents the holy grail of computer science and almost every computer science curriculum will include modules to cover the state-of-the-art in AI tools, techniques, algorithms and applications. However recently there has been a subtle but important shift in the way that humans and machines can combine their efforts to solve challenging problems. Instead of developing stand-alone, black-box AI systems capable of complex problem solving, planning, learning, and even perception, researchers and engineers have recognised the power of the Internet as a platform for so-called collective intelligence. Collective intelligence refers to scenarios where humans and machines can combine their different skills to solve problems that are otherwise beyond the research of either humans or computers in isolation.

This course will explore the core ideas underpinning collective intelligence, providing concrete examples of how some of the most complex problems (e.g., understanding images, reading text, translating speech, recognising relevant information, answering questions, even predicting future events etc) can be successfully solved by harnessing the collective intelligence of the Web. The course will also consider how the business of the Web has adapted to take advantage of collective intelligence with special attention paid to some of the emerging business models that have developed as a result.

Course Outline

This course is delivered as 12×2-hour lecture-slots and the outline below reflects the course structure as of January 2016. The course is assessed using continuous assessment in the form of 2 major implementation projects that run in parallel to the lecture series.

Lecture 1 & 2 – Introduction and Scene Setting (pdf)

  1. Housekeeping – Timetables, course structure, assessment, practicals, expectation setting.
  2. Vox Populi – An early example of crowd wisdom.
  3. Examples of collective intelligence, human computation, and crowdsourcing.
    • Crowdsourcing
    • Grid Computation
    • Kasparov vs the World
    • The PolyMath Project
    • Citizen Science (Galaxy Zoo, NASA’s Clickworkers, Open Dinosaur Project, eButterfly, …)
  4. Reading List

Lecture 3&4 – The Foundations of Collective Wisdom

  1. Framing collective intelligence
    • From AI & Data Intelligence to Crowdsourcing & Collective Intelligence
    • The Collective Intelligence Landscape
    • An architecture for collective wisdom (inputs, outputs, recruitment, …)
    • The Great Jelly Bean Jar Experiment – A concrete prediction task.
  2. Methods of Human Judgement Aggregation
    • Mathematical aggregation
    • The Condorcet Jury Theorem
    • The Doctrinal Paradox, marketplaces, …
    • The Diversity Prediction Theorem
  3. Crowd Wisdom vs. Group Think
    • Conditions for collective intelligence, deliberation groups vs prediction markets
    • Incentives and motivations.

Lectures 5-9 – Recommender Systems + Project

  1. An introduction to collaborative filtering.
  2. Rating prediction vs recommendation ranking.
  3. User-based collaborative filtering.
  4. Improving user-based collaborative filtering.
  5. Item-based collaborative filtering.
  6. Crowdsourcing & recommendation – The Recommendation Game

Lectures 10-15 – Games with a Purpose + Project

  1. Gamplay as a collective intelligence incentive.
  2. Classical GWAPs
    • The ESP Game
    • Tag-a-Tune
    • The Matching Game
  3. A Framework for Understanding GWAPs
  4. Some more GWAPs
  5. From image labelling to protein folding and beyond.
    • The FoldIt case-study
  6. Some thoughts on scaling Collective Intelligence.

Lectures 15 & 16 – Web Intelligence

  1. Accidental Collective Intelligence.
    • The Turing Test & Being Human
    • A Cryptographer does AI – CAPTCHAs
    • From spammers to scanners – ReCAPTCHAs
  3. The predictive power of search.
    • The Google Zeitgist
    • The rise and fall of Google Flu Trends.
    • Using search queries to discover drug interactions.
  4. The sentimental web.
    • Sentiment analysis and the social web.
    • Opinions & polls.

Lectures 17 & 18 – Social Information Discovery

  1. Foundations of search
  2. Social & collaborative search
  3. Reputation & trust

Lectures 19 & 20 – Participatory Sensing

  1. Collective Intelligence for the public good?
  2. From phone battery temperature to hyper-local weather prediction.
  3. Asthma and air quality.
  4. Crowd-sourced sports coverage (Run. Spot. Run., Crowdsourcing in the Field.)
  5. Incentives & Privacy

Lectures 21 – Prediction Markets

  1. Economic incentives for Collective Intelligence.
  2. Competitions & Prizes
    1. The NetFlix Prize
    2. Kaggle
    3. Innocentive, etc.
  3. Prediction Markets
    • DARPA’s “Terrorism Futures” and other examples.
    • Types of prediction markets. (payoffs and contracts).

Lecture 22 – Wrap Up

  1. Future Topics
    • Bitcoin & the Blockchain.
    • Public Health & Disease Surveillance.
    • Designing for Collective Intelligence.
  2. Conclusions

Reading List

  1. Vox Populi (Nature, 1907)
  2. Human Computation: A Survey & Taxonomy of a Growing Field (CHI, 2011)
  3. Towards a Classification Framework for Social Machines (SOCM, 2013)
  4. Run spot run: capturing and tagging footage of a race by crowds of spectators (CHI 2015)
  5.  Crowdsourcing in the Field (HCOMP, 2015)
  6. Crowdsourcing Synchronous Spectator Support (CHI 2015)

Other Resources

I maintain a collection of articles on collective intelligence as a Flipboard magazine.

Popular Books

On the Web