Winner winner vegan dinner! Twitter scraping as Q-Step interns

The UK Data Service is pleased to have had two Q-Step interns working in the Computational Social Sciences (CSS) team, based in the Cathie Marsh Institute at the University of Manchester. In this post, Julia Kasmire, head of the CSS team, introduces the Q-Step programme, and undergraduate students May and Sima reflect on what they got out of their internships.

Enhancing quantitative research skills through Q-Step

The Q-Step programme is a response to the problematic shortage of quantitative knowledge and experience within the social sciences. As a large-scale strategic programme, Q-Step gives undergraduate students a chance to increase their confidence and capacity around quantitative statistics, data, research methods and social science skills through popular features like the annual internships.

In June 2021, we were really pleased to hire May and Sima as interns in the Computational Social Science (CSS) training team at the UK Data Service. We tasked them with moving through an entire computational social science research project with only one condition – they had to use social media data!

May and Sima worked together to measure the relationship between select geographical locations in the UK and attitudes toward plant-based foods. Specifically, this meant capturing tweets with key hashtags (e.g. ‘plant-based’ and ‘vegan’) from Twitter users in London, Manchester and Preston during ‘Veganuary 2019’. They then used natural language processing and sentiment analysis to capture the general balance of positive, negative and neutral terms within the text of each tweet. Usefully, the selected time period captured a lot of Twitter discussion about Greggs’ new vegan sausage roll, including Piers Morgan’s derisive reaction to it and many peoples’ scathing reactions to him.

May and Sima’s work showed that many more people in London tweeted about plant-based diets and foods than did people in Manchester or Preston. This is not entirely surprising, given the population differences between the chosen locations. More interestingly, both London and Manchester showed a neutral-to mildly positive overall sentiment while Preston showed only a mostly neutral average sentiment. Their results suggest that most people in diverse contexts are at least neutral if not mildly positive about a “controversial” topic on a “divisive” platform. More research might tease out whether the differences relate to population density, northernliness, or something else. Scraping Veganuary 2019 tweets from Newcastle or Glasgow, for example, might help tease apart the patterns more.

Both of the interns benefited from learning to plan and run research projects, right from learning how to develop good quantitative research questions through how to collect social media data and finally arriving at how best to analyse the data and present their conclusions in accessible ways. Neither had ever done so before, so it was a real learning experience!

The CSS team benefited from a close-up observation of how social science researchers learn to think and work in more computational ways.

We also benefited from having completely new collaborators that brought their own perspectives, interests and instincts. In all, we are delighted with how much everyone involved learned  – not only about how to use social media data for social science research but also about how learning changes the way the learner approaches topics and understands new information.

Learning a new programming language: May Piskin on being a Q-Step intern

Headshot of May Piskin, a Q-Step internI was initially interested in applying for the Q-Step internship because I wanted to get some relevant, professional work experience for my CV, to increase my employability and make me stand out when applying for graduate positions. With my last year of university coming up, I also hoped that the internship would help me with my dissertation by improving my data and research skills. I was lucky enough to get accepted onto the scheme during my second-year summer holiday – this was during the Covid-19 pandemic, so I ended up doing the internship working remotely from home.

As an intern, I worked on a social media project for the Computational Social Science team. A key part of this project involved learning a new programming language (Python), which I really enjoyed. I now know a lot about RStudio and Python programming languages, which is a real benefit as I can use these skills in future employment.

Another exciting part of the project was working with Twitter data and scraping tweets. Working with Sima (another intern), I analysed and created visualisations for our final report and presentation. During my time at university I had used a few of the UK Data Service’s datasets, but I had no idea it was possible to gather and use data from social media.

I really enjoyed this process and it has taught me something I would never have learned had I not completed the internship.

Working remotely, this role challenged me to meet new people via Microsoft Teams and learn how to talk on the spot in front of others on camera. Other achievements from my time as an intern include: being able to make use of my report writing skills and design skills (which I don’t get to use in my degree); learning to organise my time better and prioritise different tasks; and improving my ability to work in a team and independently when completing objectives. The internship has also given me the chance to work creatively with social media (a topic I’m really interested in) and therefore develop my creative skillset, which I didn’t have before this internship.

Overall, this internship has given me the opportunity to collaborate on an interesting and creative research project and has given me insights into the data-driven research process. This experience has increased my confidence going into my final year at university and has given me valuable skills that I can use to complete my dissertation. The internship has also opened my eyes to the numerous graduate research positions that are available, and it has given me a strong talking point for interviews, which will likely increase my employability. I now feel very confident going out into the job market!

Gaining new web scraping skills: Sima Aykin on being a Q-Step intern

Headshot of Sima Aykin, a Q-Step internDuring the second year of my degree, I completed a Social Statistics module on survey methods, which introduced me to working with datasets and statistical software (SPSS). I found this module very interesting and knew I wanted to apply what I had learnt in a work environment. Luckily I was accepted for the Q-Step internship and got to work with the UK Data Service on a social media data analysis project over the summer.

My main aim in applying for the internship was to gain professional work experience and learn new skills. I also wanted to see if I was interested in social research as a potential career option. In addition, I knew I was going to do a dissertation project in my final year, and the skills offered by the data- and research-focused internship would prepare me well for this.

Through being involved in this project, I improved both my professional and interpersonal skills, which will benefit me in the future. Prior to the project, I was quite intimidated by working with a group of people in an online setting. But after being welcomed into the team on my first day of work, I was very happy and motivated to work with the other intern, May, and my project leaders Joe and Julia. May and I worked closely during the eight weeks of the internship, which taught me how to work as a team and communicate effectively with colleagues. I’ve also learnt how to manage my time better and deliver the projects/tasks I was given.

One of the most exciting things I’ve done in this internship was learning a new programming language (Python). Our project leader Joe gave the interns weekly ‘Coffee & Computers’ sessions where we learnt the basics of writing in Python and how we could apply these skills to our project. On top of that, I enrolled on a Python 3 course on Code Academy which was very insightful.

Prior to the internship, I had no experience of web scraping, so I learnt completely new skills when being shown how to scrape Twitter data and use it for our research project. I also enjoyed writing up our results, creating graphs and doing data visualisation for our presentation and a short report. It was a very creative and reflective process which I thoroughly enjoyed.

I’m very grateful that I had the opportunity to work for the UK Data Service during my summer break. I feel lucky to have met and worked with so many friendly and encouraging people, which improved me in so many aspects. Since completing my internship, I feel much more confident undertaking a quantitative research project, and I now have more of an idea of what I enjoy and what I want to pursue after I graduate.

I would definitely recommend the Q-Step internship to any social science students wanting to improve their social research and quantitative skills.


Julia Kasmire researches and teaches on how to use new forms of data for social scientists with the UK Data Service and the Cathie Marsh Institute at the University of Manchester.

She approaches this task as an interesting combination of thinking like a computer (essential for data sciences) and thinking like a human (essential for social sciences) in the context of complex adaptive systems. She is deeply committed to equality, diversity and inclusivity and is currently dabbling with stand-up comedy as a form of science communication.

Rumeysa (May) Piskin is a final-year BA Criminology student at the University of Manchester.

She is currently applying for graduate positions in Manchester and is confident that her internship will help her secure a job she’ll enjoy.

Sima Aykin is a final-year BSocSc Sociology student at the University of Manchester.

She is currently working on her final-year dissertation and is using the quantitative research skills gained during her internship. She’s also applying for graduate positions and looking into social data science MA programmes.


Featured image by Brett Jordan on Unsplash.

Leave a Reply

Your email address will not be published. Required fields are marked *