Scholars Access 38 Million URL Social Media Dataset, Possible Through Collaboration Between Independent Researchers and Private Sector
Social Science One (SS1) today announced the availability of the URLs Full Dataset from Facebook that will enable an array of researchers to study more about the role social media plays in democracies and civil engagement. The scholar-led organization—established in 2018 to build unprecedented partnerships between academics and data-rich private tech companies—will allow and support the use of enormous stores of anonymized Facebook data for independent, peer-reviewed social science research.
To produce this dataset, Social Science One, led by Professors Gary King (Harvard) and Nate Persily (Stanford) partnered with Facebook to process about an exabyte (one quintillion bytes or a billion gigabytes) of its data. The dataset being made available to academic researchers is protected by the principles of differential privacy and should enable informative scholarship through one of the largest datasets ever constructed for research. It includes data on 38 million URLs that have been shared publicly in more than 25,000 categories.
Director of Technology and Innovation for the Charles Koch Institute Jesse Blumenthal issued the following statement:
“Social Science One’s work to make this massive dataset available to researchers is an important step in addressing society’s most important questions about social media’s role in civic discourse and the open exchange of ideas. We believe that independent scholarly analysis of social media platforms is crucial to understanding the impact these platforms have on civic engagement and democracy around the world. The commitment of all parties involved to sharing this data in a privacy protective way presents ample opportunity for more scholarship on questions of digital free speech and the role of digital intermediaries. We’re proud to stand alongside a diverse coalition of partners in making this possible.”
In April 2018, Social Science One and Facebook launched a project on “the effects of social media on democracy and elections,” offering researchers privacy-preserving access to Facebook’s data through the structure described above.
SS1’s approach enables academics to analyze and use the increasingly rich troves of information amassed by companies to address societal issues, while protecting their respective interests and ensuring the highest standards of privacy and data security. Funding for the experimental initiative came from seven diverse nonprofit organizations including the Charles Koch Foundation Laura and John Arnold Foundation, Democracy Fund, William and Flora Hewlett Foundation, John S. and James L. Knight Foundation, Omidyar Network, and Alfred P. Sloan Foundation.
The initiative reflects the Charles Koch Foundation’s broader work supporting scholars in the pursuit and application of meaningful research.