This post highlights findings from our recent ACM CCS 2020 paper “Censored Planet: An Internet-wide, Longitudinal Censorship Observatory”, where we analyze 20 months worth of remote censorship measurement data collected in 220 countries. Censored Planet finds evidence of increasing censorship in over 100 countries, and we hope that the continued publication of Censored Planet data will enable researchers to continuously monitor the deployment of network interference technologies, track policy changes in censoring nations, and better understand the targets of interference.

High-level takeaways

• Censored Planet continuously and remotely tests reachability to more than 2,000 websites each week from over 95,000 vantage points around the world on 6 Internet protocols. Censored Planet has collected more than 21.8 billion data points in over 20 months of operation, covering networks in 221 countries (42%-360% increase compared to state of the art).
• Using time series analysis techniques, Censored Planet reports 15 key censorship events over 20 months, 10 of which were previously not reported, including in reportedly ‘‘free’’ countries. Alarmingly, in many of these events, news websites, and social media domains are increasingly blocked.
• Censored Planet detects increasing trends of DNS censorship in more than 100 countries. We also find that censorship of encrypted traffic is also increasing, showing that HTTPS alone is not sufficient to evade censorship.
• Qualitative reports such as the Freedom on the Net which only cover a few countries can be significantly complemented through data-driven reports from Censored Planet.
• In addition to longitudinal measurements, the modular design of Censored Planet enables rapid focus measurements, where we can analyze certain censorship events in more detail. We explore a case study of rapid focus measurements in Turkmenistan, where we find aggressive IP blocking of Cloudflare services.
• Censored Planet data is continuously made public at www.censoredplanet.org/data/raw, and we hope that it can enhance Internet Freedom by helping to bring transparency to censorship practices and supporting research that seeks to protect the human rights of Internet users around the world.

Background

Internet censorship, the phenomenon where access to certain websites are blocked, is a global, evolving issue, and therefore should be investigated as such. However, measuring Internet censorship globally and continuously is an extremely challenging problem – Censorship policies are intentionally opaque, diverse in their timings, methods, and targets, and may vary across space and time. Established efforts to measure Internet censorship rely on volunteers or access to machines inside a country of interest, which lead to limitations regarding scale, coverage, continuity, and safety.

A recent line of measurement techniques, called remote measurements, have paved the way to measure censorship more scalably and continuously without requiring volunteers or access to machines in specific countries. These techniques, namely Augur, Satellite/Iris, Quack, and Hyperquack, use side channels in Internet protocols to determine whether two hosts on the Internet can talk to each other, without any control over either of these hosts. We use these remote measurement techniques (which can measure Internet censorship on 6 common Internet protocols - TCP, DNS, Echo, Discard, HTTP, HTTPS) as the base for building the Censored Planet observatory.

The Censored Planet Observatory

Figure 1. (Left) Remote Measurements from the University of Michigan to vantage points across the world. (Right) Censored Planet’s modular design.

The Censored Planet observatory is a global and longitudinal censorship measurement platform that collects censorship data using multiple remote measurement techniques and analyzes the data to create a more complete view of global censorship. The Censored Planet Observatory operates from the University of Michigan, sending measurements to more than 95,000 vantage point machines across the world on 6 Internet protocols (Figure 1 (left)). Each week, the observatory tests reachability to more than 2,000 websites. Censored Planet adopts a modular design (Figure 1 (right)) that allows it to efficiently measure Internet censorship evolution using continuous repeated measurements, as well as dynamically measure certain censorship events in depth, an ability we call Rapid Focus. Censored Planet also includes methods to remove false positives from data, obtain a representative metric for censorship in each country, and time series analysis methods to detect anomalous censorship events and trends. Censored Planet releases all of the data collected publicly so that the community can use the data to track Internet censorship.

Figure 2. (Left) Censored Planet’s vantage points in March 2020. (Right) Selection of vantage points over time (Error bar shows continuity of selection).

In our ACM CCS 2020 paper, we explore Censored Planet data collected over 20 months (August 2018 - March 2020), during which the platform collected more than 21.8 billion data points. Censored Planet increases the scale of traditional censorship measurement platforms by covering networks in 221 countries (Figure 3 (left)), a 42-360% increase. Censored Planet also offers additional coverage by collecting measurements from a median of 8, and a maximum of 9,000 Autonomous Systems per country, with a median increase of 4-7 Autonomous systems per country. By not relying on volunteers and machine access, Censored Planet can reliably collect repeated measurements from the same vantage points over time (Figure 3 (right)).

Findings

Censorship Events

Table 1. (Left) Key censorship events discovered by Censored Planet. The last 10 events were previously unreported.

Censored Planet detected 15 key censorship events around the world using its anomaly detection method, augmented by manual confirmation (Table 1). Ten of these events were previously unreported. These events lasted for different time periods, and were caused by different censorship methods, showing the need for diverse and continuous measurements. Social Media and News domains were targeted in most events, while Human Rights domains were also affected. While Censored Planet detects increases in censorship around the period of important political and social events, note that it cannot identify any direct connection to these events. It’s also important to note that it’s not always government-mandated network censorship that leads to websites being unreachable, Censored Planet also detects increases in censorship over smaller networks.

Censorship Trends

We were able to find increasing levels of DNS censorship in more than 100 countries, the highest being China and Turkmenistan. Our findings show that censorship of encrypted traffic is increasing due to advanced deep packet inspection technology, which is a cause for concern. Our trend analysis also found 11 categories of domains being increasingly blocked, including News Media, Provocative Attire, Human Rights, and Gaming domains.

Complementing qualitative reports

We compared Censored Planet findings to an annual qualitative report on Internet Freedom released by the Freedom House organization called Freedom on the Net. This report has been instrumental in the Internet Freedom field and has been used by many studies to select countries for measurements. Through our comparison, we found significant censorship in countries labeled as Free by the Freedom House report. Moreover, the Freedom on the Net 2019 report covered only 65 countries, and we observed increasing censorship trends in countries not considered by Freedom House. Our findings show that quantitative results from Censored Planet can significantly complement qualitative reports like Freedom on the Net.

Rapid Focus - Turkmenistan

Getting an indication from Google Jigsaw that several DNS-over-HTTPS (DoH) were being aggressively IP blocked in Turkmenistan, we performed rapid focus Augur (TCP/IP) measurements to 34 vantage points in Turkmenistan, testing reachability to DoH services and Cloudflare IPs. Our findings confirmed that 52.9% of vantage points in Turkmenistan blocked all Cloudflare IPs, restricting access to thousands of services. Other DoH services (e.g. Snopyta) were also blocked.

Conclusion

Our results demonstrate Censored Planet’s ability to create a more complete picture of global censorship that is complementary to existing platforms and qualitative reports. We will continue to publish Censored Planet data which will enable researchers to continuously monitor the deployment of network interference technologies, track policy changes in censoring nations, and better understand the targets of interference. Ultimately, we hope that making opaque censorship practices more transparent at a global scale counters the proliferation of these growing restrictions to online freedom.