The next speaker in this panel at the AoIR 2025 conference is my QUT colleague Laura Vodden, presenting her work on exploring LLM-assisted frame analysis of news coverage. This focusses here especially on Australian climate activism news coverage. The first challenge here, of course, is to understand framing, which usually includes a problem definition, suggested causes, proposed solutions, blame attribution, and and addressee for the solution. Such framing frequently occurs in news reporting.
Laura’s slides are here:
Frame analysis is a difficult and labour-intensive task, however; it requires critical engagement with complex material, and human coding is highly time- and resource-intensive and therefore usually restrained in scope to small-scale analyses. It is also affected by the coders’ own personal and political biases and assumptions. The question, then, is whether the use of computational tools like LLMs can assist with such work.
Often such work is limited to language analysis or topic modelling, which does not capture the full depth of framing; such analysis can serve as proxies for frames, but do not themselves represent full frame analysis. Is there a way to extend such work to develop more fully-featured frame analysis approaches, then?
A substantial part of the literature on this takes a deductive approach, defining analytical categories beforehand and then seeking to identify these pre-defined frames in the data; the present project takes a more inductive approach that seeks to allow the frames to emerge from the data analysis itself. The focus here is on climate protest coverage from January 2019 to September 2023, focussing on protests ranging from the Fridays for Future and School Strike for Climate movements to the more radical Extinction Rebellion.
The approach here works with a group of human experts and LLMs coding articles in parallel. Both inductively sought to classify the frame elements in such data (problem definition, causes, etc.), which were then synthesised into larger frames. The project then compared these human- and LLM-derived frames, seeking to refine the frame definitions to a point where they might be used for further deductive coding.
However, this process also showed up an issue with the human coding: the human coders themselves varied widely in how they identified, extracted, and described the frame elements they were asked to focus on; humans were often considerably more verbose and specific in what they identified. In spite of such variations, the underlying concepts extracted by humans and LLMs were fairly similar, though, and a synthesising of these initially identified specific frame elements into more summary descriptions shows considerably levels of similarity after all.
There is therefore substantial potential for the use of LLMs in frame analysis, but frames are a complex concept, and describe a relationship between the author and reader of a text, and this needs to be taken into account. Very specific coding here is counterproductive, therefore, as it introduces very individualistic readings that are not useful for broader categorisations; what is necessary here is to distinguish genuine coder disagreement from mere differences in coding styles.
