Resource Library

Welcome to the TSPA Resource Library. We use this space to collect articles, papers, lectures, and podcasts that trust and safety professionals may find useful in developing policies, supporting moderators, building systems to detect violations, and generally deepening their practice.

We’re working on making this library searchable; in the meantime, you can browse resources by category. We welcome your submissions and feedback.

This collection is a project led by Eric Davis and the TSPA Resource Development Working Group, with special thanks to Paul M. Barrett, Eric Goldman, and Danielle Leong.

TSF’s case study library can be accessed here.

Featured resources added this month:

Crafting and iterating policies

TOP
Policy Development & Launch ChecklistSidequest
Facebook Product Policy Forum Minutes
How does Patreon create content policies?Patreon, 2019

Enforcing policies

TOP
Instagram’s Kevin Systrom Wants to Clean Up the &#%$@! InternetNicholas Thompson, Wired, 2017
YouTube: An update on our commitment to fight terror content onlineYouTube, 2017
YouTube Official Blog, Expanding our work against abuse of our platformYouTube, 2017

Community management

TOP
discord moderator academy illustration - cartoon books and study materials
Discord Moderator AcademyDiscord
community signal logo
Community Signal PodcastCommunity Signal
Open Source Survey(Github Open Data Project, 2017)
Community Interaction and Conflict on the Web(Stanford researchers, 2018)
Codes of Conduct: Maybe They’re Not So Bad?
Social media governance: Can companies motivate voluntary rule following behavior among their users

Examining issues and drivers

TOP
green inhope graphic saying "what is..?"
What is… Terminology and Technology Related to Child Sexual Abuse Material (CSAM)INHOPE
Haters: Harassment, Abuse, and Violence Online(Bailey Poland, published 2016)
Autobiography of a Recovering Skinhead(Frank Meeink, published 2010)
Alt right glossary(RationalWiki)
Hate symbols database(Anti-Defamation League)
The Aesthetics of the Alt-Right (M. Ambedkar, Post-Office Arts Journal, 2017)
The Rhetorical Dynamics of Gender Harassment On-Line (academic research)(Indiana University researcher, 1999)
Online Harassment(Pew Research, 2014)
Online Harassment 2017(Pew Research, 2017)
Crossing the line: What counts as online harassment?(Pew Research, 2018)
1 in 3 Americans Suffered Severe Online Harassment in 2018(Anti-Defamation League, 2019)
Nobody’s Victim: Fighting Psychos, Stalkers, Pervs, and Trolls(Carrie Goldberg, Published 2019)
The Internet of Garbage(Sarah Jeong, published 2018)
Are sex differences in antisocial and prosocial Facebook use explained by narcissism and relational self-construal?(University of London researchers, 2017)
Gender differences and bias in open source: Pull request acceptance of women versus men(California Polytechnic State and North Carolina State researchers, 2016)
The Real Name Fallacy (J. Nathan Matias, the Coral Project by Vox Media, 2017)
Cross-platform disinformation campaigns: lessons learned and next steps(Tom Wilson and Kate Starbird, Harvard Kennedy School Misinformation Review, 2020)
Disinformation and 2020 Election: How the Social Media Industry Should prepare(Paul Barrett, NYU Stern Center for Business and Human Rights, 2019)
Verification Handbook for Disinformation And Media Manipulation (Edited by Craig Silverman, 2020)
We can all help with Overwatch’s toxicity — but Blizzard must lead(Frank Sanchez, VentureBeat, 2017)
The Simple Psychological Trick to Political Persuasion(Olga Khazan, the Atlantic, 2017)
How Trolls Are Ruining the Internet(Joel Stein, Time, 2016)
Anyone can be a troll(Cornell and Stanford researchers, TheConversation.com, 2017)
The paradox of tolerance(Wikipedia)
Dissecting Trump’s Most Rabid Online Following(Trevor Martin, FiveThirtyEight, 2017)
Escape the echo chamber(C Thi Nguyen, Aeon, 2018)
How Community Feedback Shapes User Behavior(Stanford researchers, 2014)
Social media governance: Can companies motivate voluntary rule following behavior among their users (Facebook/academic research

Content Moderation Approaches and Challenges

TOP
Volunteer Moderators in Twitch Micro Communities: How They Get Involved, the Roles They Play, and the Emotional Labor They ExperienceCHI '19 (Wohn)
new america logo
Everything in Moderation: An Analysis of How Internet Platforms Are Using Artificial Intelligence to Moderate User-Generated ContentNew America, 2019
All Complex Ecosystems Have Parasites(Cory Doctorow, personal site, 2005)
This is How to Build a Troll-Free Online Community(Ryan Paugh, FastCompany, 2018)
How do you set metrics?(Julie Zhuo, personal blog, 2017)
Internet Content Moderation 101(Hunter Walk, personal blog, 2017)
Google’s Gatekeepers(Jeffrey Rosen, the New York TImes, 2008)
Why The History Of Content Moderation Matters(Kate Klonick, Techdirt, 2018)
The Secret Rules of the Internet(Type Investigations, 2016)
The Laborers Who Keep Dick Pics and Beheadings Out of Your Facebook Feed(Adrian Chen, Wired, 2014)
The Moderators(Documentary by Field of Vision, 2017)
Content Moderation at Scale (COMO) Summit(Santa Clara Law, 2018)
Archives from Content Moderation & Removal at Scale conference(Santa Clara Law, 2018)
26 recommendations on content governance: a guide for lawmakers, regulators, and company policy makers(AccessNow, 2020)
You Can’t Stay Here: The Efficacy of Reddit’s 2015 Ban Examined Through Hate Speech(Georgia Institute of Technology, Emory University and University of Michigan researchers, 2017)
Anonymity and Abuse Reports(Cloudflare, 2017)
Instagram’s Kevin Systrom Wants to Clean Up the &#%$@! InternetNicholas Thompson, Wired, 2017
How Facebook Tries to Regulate Postings Made by Two Billion PeopleWorkshop at Berkman Klein Center for Internet and Society, 2017
Facebook: Hard Questions: How We Counter TerrorismFacebook, 2017
YouTube: An update on our commitment to fight terror content onlineYouTube, 2017
YouTube Official Blog, Expanding our work against abuse of our platformYouTube, 2017

Resources for Users

TOP
HackBlossom
Staying Safe When You Say #MeToo(ACLU)
Technology Safety(Safety Net Project by the National Network to End Domestic Violence)
Security Checklist
Speak Up & Stay Safe(r)
EFF’s Surveillance Self-Defense
Surveillance Self-Defense Checklist
Freedom of the Press Foundation Guides and Training
Smart Girl’s Guide to Privacy(Violet Blue, published 2014)
How to Opt Out of the Sites That Sell Your Personal Data(Wired, 2019)

Building Systems for Addressing Violative Behavior

TOP
flowchart illustrating how machine learning develops
Machine Learning for Content Moderation — IntroductionDevin Soni, 2019
illustration of pinterest's machine learning approach
How Pinterest fights misinformation, hate speech, and self-harm content with machine learningPinterest Engineering, 2021
Trust & Safety Engineering @ GitHub(Video, Lexi Galantino, 2019)
Trust and Safety Lessons for New Founders and Engineers(Video, Craig Newmark in discussion with Alex Stamos at Stanford Cyber Policy Center, 2019)
Natural Language Processing: Paying attention to toxic comments online(Stanford researchers, 2018)
Trust and Safety Engineering course, Stanford University

Identity Verification and Account Protection

TOP
Authentication Best Practices(Open Web Application Security Project, a.k.a., OWASP)
Easy Ways to Build a Better P@$5w0rd(Updated NIST password guidance, 2017)
Design Best Practices for an Authentication System – IEEE Cybersecurity
Evaluating Login Challenges as a Defense Against Account Takeover(Google Research, 2019)
Usage of Behavioral Biometric Technologies to Defend Against Bots and Account Takeover Attacks(Ajit Gaddam, Visa, Usenix Enigma, 2019)
Anatomy of Account Takeover(Grzegorz Milka, Google, Usenix Enigma, 2018)

Modeling and Planning

TOP
product school logo
Creating safe digital experiences (Product School presentation)Ram Ravichandran and Sreekant Vijayakumar, 2019

Transparency and accountability

TOP
new america logo
The Transparency Report Tracking ToolNew America, 2021
Santa Clara Principles on Transparency and Accountability in Content Moderation(Content Moderation at Scale conference in Santa Clara, CA, 2018)
AccessNow’s Transparency Reporting Index
Google’s Transparency Report
Facebook’s Transparency Report
Twitter’s Transparency Report
Wikipedia’s Transparency Report
Reddit’s Transparency Report

Evaluating Transparency and Metrics

TOP
The Abuse Uncertainty Principle, and Other Lessons Learned from Measuring Abuse on the Internet(David Freeman, Usenix, 2020)
Facebook Data Transparency Advisory Group Releases Final Report(Data Transparency Advisory Group (DTAG) at Yale Law School’s Justice Collaboratory, 2019)
Facebook discusses transparency group report’s findings(Facebook, 2019)
Facebook’s Information quality processes for Community Standards Violations metrics(Facebook, 2019)

Examples of community guidelines and other policies

TOP
YouTube Community Guidelines
Xbox Community Standards
Vimeo Community Guidelines
Verizon Media Ads Guidelines
Verizon Media Community Guidelines
The Verge Community Guidelines
Twitter Ads Guidelines
Twitter Community Guidelines
Twitch Community Guidelines
Tumblr Ads Guidelines
Tumblr Community Guidelines
Tinder Community Guidelines
TikTok Ads Guidelines
TikTok Community Guidelines
Steam Community Guidelines
SoundCloud Community Guidelines
Spotify Community Guidelines
Snap Ads Guidelines
Snap Community Guidelines
Skillz Community Guidelines
Scratch Community Guidelines
Reddit Community Guidelines
Quora Ads Guidelines
Quora Community Guidelines
Pinterest Ads Guidelines
Pinterest Community Guidelines
Patreon Community Guidelines
OkCupid Community Guidelines
Nextdoor Community Guidelines
Mozilla Community Guidelines
Microsoft Ads Guidelines
LinkedIn Ad Guidelines
LinkedIn Community Guidelines
Kickstarter’s Community Guidelines
Instagram’s Community Guidelines
Google’s Ads Policies
Glassdoor’s Community Guidelines
GitHub’s Community Guidelines
Fandom’s Community Guidelines
Facebook’s Ads Policies
Facebook’s Community Guidelines
Eventbrite’s Community Guidelines
Etsy’s Community Guidelines
Discord’s Community Guidelines
Apple’s Ads Policies
Apple’s App Store Guidelines
Amazon’s Ads Guidelines
Amazon’s Community Guidelines

Critiques of policies and governance

TOP
Consent of the Networked(Rebecca MacKinnon, Published 2013)
Meet the Man who Deactivated Trump’s Twitter Account(Ingrid Lunden and Khaled “Tito” Hamze, Tech Crunch, 2017)
Facebook’s Uneven Enforcement of Hate Speech Rules Allows Vile Posts to Stay Up(Ariana Tobin, Madeleine Varner, and Julia Angwin, Propublica, 2017)
Facebook’s Secret Censorship Rules Protect White Men from Hate Speech But Not Black Children(Julia Angwin, ProPublica, and Hannes Grassegger, special to ProPublic, 2017)
The Facebook Files(the Guardian, 2017)
The Pollyannaish Assumption(Ben Thompson, Stratechery, 2017)
The Limits of Automated Social Media Content Analysis(Center for Democracy and Technology, 2017)
Controlling the Conversation: The Ethics of Social Platforms and Content Moderation(Priyanjana Bengani, Tow Center for Digital Journalism, Columbia University, 2018)
Who Moderates the Social Media Giants?(Paul Barrett, NYU Stern Center for Business and Human Rights, 2020)

Public policy and law

TOP
The Twenty-Six Words That Created the Internet(Jeff Kosseff, Published 2019)
Who Do You Sue? State and Platform Hybrid Power Over Online Speech(Daphne Keller, Lawfare, 2019)

Wellness and resilience considerations

TOP
Fast, Accurate, and Healthier: Interactive Blurring Helps Moderators Reduce Exposure to Harmful ContentDas, Dang, and Lease (2020)
But Who Protects the Moderators? The Case of Crowdsourced Image ModerationDang, Riedl, and Lease (2018)
Testing Stylistic Interventions to Reduce Emotional Impact of Content Moderation WorkersKarunakaran and Ramakrishan (2019)
CHI 2021 conference logo
The Psychological Well-Being of Content ModeratorsCHI ’21 (Steiger, Bharucha, Venkatagiri, Riedl, and Lease)