This page is optimized for a taller screen. Please rotate your device or increase the size of your browser window.

The Role of #SocialMedia in #PolicyResearch

December 28, 2015
Social media is replete with promotions, half-truths, and blatant falsehoods. Pith triumphs over validity and nuance. So social media seems like the last place to advance the cause of rigorous social, economic, environmental, or health policy research, which is at the core of Abt Associates’ mission.

But an ever-increasing number of people live and work digitally and on social media. The Pew Research Center reports that fully 62 percent of the U.S. adult population uses Facebook, and of those users, 70 percent use it daily. Meanwhile Twitter boasts 315 million users – roughly equivalent to the U.S. population – who generate more than 500 million tweets per day.

Therefore, policy researchers should venture into the social media badlands and harness it to advance research. Meanwhile, we need to learn from social media and maybe contribute to improving its overall character.

In this spirit, here are seven ways researchers can engage with social media. Below that is a discussion of some of the pitfalls.

  1. Direct interaction with the public: Social media may not always be the best place to find verifiable answers, but it can be a great resource for defining the questions and seeking input. Through social media, and with the right types of disclosures, researchers can interact directly with the intended beneficiaries of a policy or those suffering from a particular health issue, perhaps via the related organizations that the target audience may follow. Social media allows beneficiaries to participate in this process, help frame the questions and drive the research agenda. Similarly it can be a source of qualitative data, such as feedback on a program’s ability to deliver a service.

  2. Pooling expertise and insights: Researchers at Abt Associates often are asked to convene expert panels to refine questions, methodologies, or analyses. This essentially is high-end crowdsourcing. This can be supplemented by broader crowdsourcing through social media affinity networks. For example, professional groups on LinkedIn could be used to conduct the discussion online, either openly or in a private group.

  3. Recruitment: One major challenge for researchers is recruiting people to participate in surveys, demonstrations or focus groups. Researchers can use social media for this initial outreach, spreading the word through personal or organizational channels of research study opportunities.

  4. Follow-up contact: Unlike telephones and email, social media are specifically designed and optimized for finding people that you have lost touch with, so researchers can actually search social media to find their missing participants and reduce study dropout rates.

  5. Social media analysis: Perhaps the most interesting and exciting potential area for leveraging social media is the analysis of social media data itself. Twitter has been a popular source of data for such research due to the openness of the content and the accessibility of the data. A popular strategy is to use “sentiment analysis” to ascertain the overall positive or negative tone of the tweet and correlate it with other data and ultimately attempt to predict real-world events. Some examples:
    • A 2010 Carnegie Mellon study found that automated analyses of sentiments expressed on Twitter regarding consumer confidence. Presidential approval and elections are strongly correlated with traditional survey results.
    • State-level Twitter sentiment data has also been shown to predictive of enrollment under the Affordable Care Act. That is, enrollment turned out to be higher in states and months where positive sentiments about the program were expressed.
    • Researchers at the University of Michigan are using Twitter data on job loss to predict unemployment claims. LinkedIn can also be a rich source of insights on job market trends.

  6. Social Media as Object of Study: The prevalence of social media makes it not only a tool for research but an object of research. For example, Facebook researchers in a partnership with Carnegie Mellon have studied the effects of weak and strong ties both on Facebook and offline after a job loss in terms of both stress levels and finding new employment. The role of social media in nudging people toward certain desired behaviors is also an important topic. Facebook’s randomized experiment in 2010, published in Nature, in encouraging users to vote and allowing them to easily share that they voted found that they were able to increase the likelihood of voting by a statistically significant margin.

  7. Dissemination: Disseminating results is the last phase of the research project but probably the primary usages that researchers think of in terms of using social media. Social media should not be a direct substitute for publication in peer-reviewed journals. But, researchers looking to influence the public discourse or implementation practices need to spread the word through more widely accessed platforms.

Privacy, Validity, and Bias Issues

Social media as a research tool also raise several areas of concern:

Privacy: Using social media as a data source raises numerous privacy questions. Even if posts used for analysis were public, participants might not have expected the content they provided might be used for research purposes. The acquisition of de-identified content or metadata from the social media company, and perhaps matching those data with other sources deepens the questions.

Human Subjects: The notion of experimentally “nudging” people toward certain desired outcomes without their consent requires strong IRB oversight. Facebook came under intense media – and social media – criticism in summer 2014 for conducting a mood experiment, that involved randomly manipulating user feeds and assessing the impact on people’s moods as determined by the content of their posts.

Validity: Many of the studies using social media as the data source are proofs of concept, intended to determine the circumstances under which social media can be a valid substitute for more traditional approaches, such as surveys or analysis of administrative data. These models still are being defined. One problem is that, at least in the short-term or in certain defined geographies, social media posts are subject to external forces in the culture. For example, tweets about a government program, such as disability benefits, may normally reflect sentiment among beneficiaries of the actual program, but may at certain times be reflecting other things in the atmosphere, such as a candidate mentioning the issue in a debate.

Bias: Using social media for both recruitment and follow-up for a study, like most other methods, introduces selection biases. While social media use, and Facebook in particular, has reached all sectors, it has not reached them equally. Age is a particularly important bias, with 90 percent of people between 18 and 29 use social media, compared with 35 percent of people over 65.

Long-term Viability: While Facebook’s fortunes seem to be growing steadily, usage among teenagers reportedly is in decline. Meanwhile, Twitter is having difficulties financially and its usage seems to have plateaued. So while social media likely is here to stay, technology trends change rapidly. One probably should not depend on a single platform or service in the competitive market over an extended period.

Despite these cautions, social media is a critical frontier for researchers and an important tool for many facets of the research process. The question is not whether to use it, but how to use it well.

Note: This is an abridged version of a post that appears on the Abt Associates Technology website. Read that version here.