Using Social Media to Evaluate Grassroots Campaigns in Real-Time: A Case Study of Twitter Users' Response Following a Social Protest

Introduction: This study aims to provide an analytical pipeline for evaluating grassroots campaigns using social media data. Engaging in grassroots campaigns is one way for citizens to promote a more just and equitable society. For example, the ‘No Ban No Wall’ campaign that took place on January 27, 2017 protested against President Trump’s executive actions on building a wall along the Mexico-US border and banning citizens of seven predominantly Muslim countries from entering the US. Evaluation of grassroots campaigns in real-time can provide immediate feedback for activists and community organizers. In this study, we describe an analytical pipeline that utilizes a quasi-experimental study design and text mining techniques, then demonstrate its effectiveness by examining changes in Twitter users’ sense of justice in immigration discussions following the #NoBanNoWall campaign.

Method: The study team collected 1,445,520 posts on Twitter from a user panel (n = 1,213) over a four-week period, with the two weeks prior and two weeks following the event comprising the before and after time intervals, respectively. We identified the user panel by extracting the user screen names from tweets related to immigration issues and specifically affected groups (Muslim and Mexican). Related tweets were defined as those that contain phrases that match any of the keyword patterns “immigra*,” “mexican(s),” and “muslim(s)”. For example, tweets containing either of the words “immigrant” or “immigration” would match the “immigra*” pattern and therefore represent discussions related to immigrants in general. For comparison purposes, we also created a baseline of tweets that did not match any of the search terms. We employed sentiment analysis to capture Twitter users’ sense of justice using the psycho-linguistic lexicon LIWC (Linguistic Inquiry and Word Count). Odds ratio (OR) were then computed to measure changes in the sense of justice on immigration issues in user tweets before and after the campaign.

Results: Compared to baseline, there was a significant increase in tweets related to immigrants in general (OR = 3.24, p =0.00) after the ‘No Ban No Wall’ campaign. While there was a significant increase in tweets related to Muslims (OR = 4.26, p = 0.00), the tweets related Mexicans were significantly decreased (OR = 0.82, p = 0.03). Compared to baseline, Twitter users’ sense of justice was significantly higher in immigrant-related tweets (OR = 2.41, p = 0.00) and Muslim-related tweets (OR = 2.86, p = 0.00) after the protest. However, this pattern was not found in Mexican-related tweets.

Implications: This study demonstrated a simple analytic pipeline that uses social media data to evaluate grassroots campaign in real-time. The findings of this study revealed that while the ‘No Ban No Wall’ protest raised the sense of justice in discussions of immigrants in general and Muslims in particular, it did not increase the sense of justice on how Twitter users discussed Mexicans. We call for social work researchers to utilize current methods of data science to increase the capacity for big data analytics in advancing the understanding of the role of grassroots campaign in promoting a just and equitable society.