Alternative data generally refers to information beyond traditional sources like corporate financial statements and filings, stock market data, and government reports. Initially used mostly by investment firms, today, government agencies also use alternative data to analyze economic, social, and political environments and formulate policies.
More information than ever before is digital, creating new, alternative data sources. This article summarizes these sources and provides examples of how governments use alternative data to implement policies aimed at improving social welfare, reducing crime rates, bolstering health care, and protecting the environment.
What Is Alternative Data?
Alternative data generally refers to information beyond traditional sources like corporate financial statements and filings, stock market data, and government reports. The use of digital applications to conduct business, governmental, and social transactions has led to an increase in the production of public data. As a result, sources of alternative data are growing.
What Are the Sources of Alternative Data?
The use of digital applications to conduct business, governmental, and social transactions — commonly referred to as “digitization” — leads to an increase in the production of public data. As a result, sources of alternative data are growing. They presently include all the following:
6 Sources of Alternative Data
- Mobile and desktop applications.
- Credit and debit card transactions.
- Email receipt transaction data.
- Satellite data.
- Low-level drones.
- Public databases and web scraping.
1. Mobile and desktop applications
Applications are a trove of data that reveal insights into consumer preferences and behaviors. Industries that collect data and use this particular source include travel, food delivery, consumer goods, health and fitness, transportation, and entertainment.
2. Credit and debit card transactions
Credit and debit card transactions reveal insights into consumer spending habits, preferences, and disposable income. Common use cases include academic, government, and marketing research.
3. Email receipt transaction data
Similar to credit and debit card transaction data, receipts sent via email to purchasers reveal critical information about their behavior and spending patterns. Subject to privacy laws and regulations, email receipt transaction data is obtained from various sources, including email service providers and third-party data firms. Common use cases include academic, government, and marketing research.
4. Satellite data
Satellites are a valuable way to collect environmental and human behavior data. Examples include measuring and monitoring vegetation, land use, coastline changes, soil moisture, surface elevation, tectonic activity, melting glaciers, temperature changes, and air quality.
5. Low-level drones
Drones equipped with cameras and sensors can capture high-resolution images, videos, and detailed data from the Earth’s surface. Applications include monitoring changes in building infrastructure, vegetation, agriculture, topography, and air quality.
6. Public databases and web scraping
The vast expanse of data available on the internet makes online public resources a dominant source of alternative data. Some examples include public databases and web scraping.
Public databases include filings from the European Securities and Markets Authority (ESMA) and Securities Exchange Commission (SEC), global supply chain import and export records, patent and trademark filings, and government contracts. Though valuable, these documents are typically challenging to obtain at a large scale due to inaccessible file formats, like PDF.
Publicly available information on social media sites, marketplaces, and other websites can provide a wide range of alternative data. Some examples include the following:
- Real-time product and service pricing data
- Consumer sentiment data from product reviews
- Stock, bond, and commodity pricing data
- Job market data
- Census data
- Real-time weather data
Online public data is typically obtained through web scraping — the automated gathering of information from the internet.
Web scraping uses scripts and software tools to extract, parse, and store data in a structured format that analysts can read. Examples of data types gathered by web scraping include text, images, and videos. Watch a live demonstration of a web scraper in action to learn more.
5 Ways Government Agencies Use Alternative Data
Government agencies are increasingly unlocking the power of alternative data to gain insights and make policy decisions that can positively impact citizens.
Recent examples include the following:
5 Policy Uses for Alternative Data
- Economic “nowcasting.”
- Social welfare programs.
- Environmental policy.
- Health care and transportation planning.
- Crime reduction.
1. Economic ‘Nowcasting’
Nowcasting — a portmanteau of “now” and “forecasting” — uses high-frequency data and statistical models to predict the present or near-term future state of the economy.
Alternative data sources such as social media, online job postings, credit card transactions, and satellites typically provide timelier and more relevant information than traditional sources like government reports. As a result, government agencies (e.g., the Central Bank of Japan) are nowcasting indicators like inflation using alternative sources, such as mobility data.
2. Social welfare programs
The Fundación Paraguaya, a self-sustainable NGO, helps people living in poverty by using alternative data to identify their needs and, based on this analysis, provide resources to help them achieve their goals.
As part of its strategy, the foundation extracts data from a dedicated application and web platform to identify economic trends, generate real-time reports, and create “poverty maps.” These activities enable policymakers to develop strategies that channel aid to areas that need more resources.
3. Environmental policy
The European Union’s Copernicus program collects air and water quality data using various methods (like satellites) to inform policy decisions. For example, satellite data collected by researchers help identify high-pollution areas. Doing so enables policymakers to make regulatory decisions aimed at reducing emissions.
4. Health care and transportation planning
In 2018, the government of Singapore launched “Smart Nation,” a program that uses technology and data aimed at improving the lives of its citizens. Part of the program involved a pilot project called “Smart Elderly Monitoring and Alert System,” which promoted the use of the Healthy 365 App to monitor the health of elderly residents living in public housing.
Another example was the on-demand public bus service trial in 2018 that paired dynamic routing and matching algorithms to optimize public bus services. Using a mobile app, commuters requested pick-ups and drop-offs at any bus stop on routes with unpredictable or low ridership.
5. Crime reduction
The city of New Orleans launched a program in 2017 called “Operation: Real-Time Crime Center.” This initiative used alternative data from several sources to identify and respond to crime in real-time, including social media, public cameras, and other surveillance technology.
According to a case study published by the agency, the program reduced murders by 46 percent between 2018 and 2019. In addition, gun violence was reduced by 82 percent in one of the most dangerous areas, and city-wide armed robberies dropped by more than 30 percent.
Alternative Data Can Positively Impact Social Change
Alternative data can come from many places, but the internet remains the largest and most accessible source of information available. You can learn more about alternative data scraping, its challenges, and how to overcome them in our whitepaper: Scraping Alternative Data: Technological Challenges to Keep in Mind.
