Abstract
Since the introduction of OpenAI’s ChatGPT, LLMs have risen rapidly in popularity, with summaries structuring not only every single Google search, but also scientific research and even social interventions. Despite the scale of deployment and the practical inescapability of LLMs today, we know little about the way models mediate information, including what narratives and political frames they prioritize, and how they choose to handle “sensitive topics” of antisemitism and islamophobia. Seeking generality, technical disciplines sidestep diving into domain specifics, leaving us with general metrics that sidestep inconvenient details. We argue, however, that LLMs are not only tools, but active contributors to knowledge creation and that we must take seriously the ways in which they reshape our discourse. Through a mixed quantitative and qualitative methodology, this study evaluates how model summaries alter the political lean, framing, and fact representation of news on Israel and Palestine and employs surveys to see how that changes the perception of antisemitism and islamophobia. By analyzing how LLM summaries alter news perception, our findings will inform policy on AI regulation and social science research, where the capacities of LLMs to summarize information is implicitly used everyday, including studies on anti-hate interventions and AI counter speech.
