It’s the first day in October, and I’ve made my way to the 5th Data Donation Symposium at the Weizenbaum-Institut in Berlin, and we start today with a keynote by Jasper Tjaden from the University of Potsdam, whose topic is social media data acquisition through social research bots. This is especially valuable for social media audit studies; it moves beyond the understanding of bots as problematic actors that meddle in information and communication flows, and instead uses bots for good.
Such bots then become a vehicle for data collection, and fight for transparency in data science. One way of doing this is by using bots for sock puppet audits: setting up bots to impersonate a real user’s persona, and observing how these automated personas are being served content by platform algorithms. This also enables researchers to test several demographic, interest, or behavioural profiles, and thereby to estimate how these attributes affect user experience on a platform.
This starts by designing a behavioural profile, creating accounts that match such behaviours, automating bot account activities, and finally extracting the feed of content the bots experience. Such data may also need to be further enriched by gathering additional metadata and scraping offplatform content; all of this is then analysed by the researcher. To gather the feed content, several options are available: screen captures of the browser interface; document object model (DOM) scraping; or network interceptions which capture the JSON data of the hidden API interactions which the bot behaviour triggers.
This last approach is used here: the bot interacts with a platform like TikTok, and through this interaction the TikTok API sends new JSON objects to the browser; this is captured by an interception layer during the bot interaction process, and stored in a feed database which is then available for research purposes
. Such approaches have been employed for instance in a study of political bias in platform feeds (in a US presidential election context); in a study that used real user histories to model bot personas during a learning phase and then either continued or diverged from this to see the difference in user experience; and in a study that created politically biased bot accounts and then used LLMs to classify the TikTok videos they encountered, finding a notable pro-Republican leaning in the US.
This approach is distinct from both API- and data donation-driven approaches; the former provides a platform-centric, aggregate, supply-side perspective, and the latter a retrospective perspective strongly influenced by the specific types of users who provide data donation, while bot-based approaches afford the researcher greater control over the (simulated) user experience as it is happening, and enable a direct comparison between distinct (artificial) user types and behaviours. However, of course, bots do not provide the researcher with an opportunity also to explore genuine users’ motivated reasoning for their activities, or reactions to the content they encountered.
A new paper Jasper and his team are now working on explores these differences more systematically: building on a large dataset of over 2,000 politicians at EU, federal, state, and regional levels in Germany who have accounts on TikTok, they are exploring what videos their bots encountered, and are comparing this with API-derived information on the videos these political accounts posted, as well as data donations from actual users.
This has found that videos by accounts from the far-right AfD have a 16.5% higher likelihood of appearing in these bots’ feeds, even though these bots did not show any particular political interests; this is not driven as much by inherent biases in the TikTok algorithm, however, but by the AfD’s much greater embrace of TikTok as a platform, and consequently their greater level of content creation.
Of course, such bot-driven data capture is potentially in violation of the platform’s Terms of Service; this is usually justified by the prosocial aims of the research, however, and Terms of Service are often legally problematic in the first place. Other problems include the fact that bot-driven research can usually only work technically within desktop browser environments, and that even the most realistic bot behaviour may still differ from that of ordinary human users.
Data donations can help here, in particular: bot behaviour could be informed by the activities captured in data donations from actual human users, for example, while the use of bots could also expand observations beyond the type of user who typically is willing to donate their activity data to researchers. Data donors might also be enticed to engage with bots, and the effects of such engagement could be analysed further.
