ChatGPT vs Claude vs Gemini: Which AI tool is best for summarising?
AI transcription services are a game-changer, making the chaotic scrawl of handwritten notes obsolete. But when every detail is captured into a Word doc, and you find yourself facing a dense wall of text with no paragraph breaks and no context, how do you discern what’s important and extract the information you need?
Today, the most practical answer is to revert to AI (again) to condense, summarise actions, pull out key takeaways, draw out quotes or anything else that could be done manually.
All of the main AI chatbots offer a ‘summary’ function, but how often do we question which is the best tool for the job? Are there core differences between outputs? Are we cross-referencing AI summaries with our own takeaways, or the original transcript itself to make sure it’s accurate?
When efficiency and time savings are the name of the game, these best practices are, more often than not, assumed instead of observed.
This is why we decided to conduct our own research and test AI tools’ ability to summarise transcripts. We used a standardised prompt, which we fed to ChatGPT, Gemini and Claude, to sum up our recent webinar on Communicating AI Responsibly: How to navigate the hype, ethical considerations and human impact.
The discussion had three experts - Aled Lloyd Owen, Professor of Enterprise and Chief of Staff at Responsible AI UK; Jasper Hamill, freelance tech journalist and founder of the publication Machine; and Susi O'Neill, author and Executive Director at EVA: Responsible AI + Digital Trust - weighing in on what it means to communicate AI’s benefits responsibly and ethically.
We asked ChatGPT (version: GPT -4.0), Gemini (version: Gemini 2.0 Flash) and Claude (version: Claude 3.7 Sonnet) to summarise the top takeaways, expert opinions, actionable recommendations and notable quotes from the webinar, and the results were… surprising.
Top Takeaways
Winner: Gemini
ChatGPT offered the broadest thematic summary, capturing general ideas from the discussion. However, this came at the cost of specificity. Its takeaways felt more high-level and generic, with few unique or striking insights. Notably, it missed two of the main themes that both Gemini and Claude identified as central to the discussion: ‘Hype vs. Reality’ and ‘Human-AI Collaboration’.
Claude stood out for depth and detail. It surfaced the most takeaways (six) and often supported them with relevant quotes and statistics, which added substance and context. The insights felt more thoughtful and precise. That said, Claude overlooked one key theme - regulation - which both ChatGPT and Gemini flagged as important.
Gemini struck a balance between breadth and relevance. It captured all the major themes from the transcript and provided a solid overview of each. While it didn’t go as deep or specific as Claude, it didn’t omit any of the key discussion points either.
Expert Opinions
Winner: Claude
When it came to summarising the most thought-provoking expert insights from the webinar, Claude stood out as the most insightful and articulate. It surfaced nuanced ideas with clarity and supported them with well-chosen pull quotes; an approach that added trust and weight to its summary. Claude also highlighted concepts that other tools missed entirely, such as the importance of anthropomorphising AI to make abstract ideas more relatable, and the dangers of the persistent “silver bullet” narrative surrounding AI. Additionally, it flagged Susi’s “WAIT” (Working on AI Transformation) acronym and referenced “neo-Luddite” responses: both important contributions to the broader conversation on AI ethics and public reception.
In contrast, ChatGPT’s summary tended to gloss over more sophisticated angles. It missed Jasper’s compelling argument that a degree of sensationalism in AI media coverage isn't inherently bad, but can be used to help capture attention and convey important ideas. Similarly, it overlooked the "AI insiders" dynamic, a fresh and valuable lens on how narratives are shaped. While it did identify trust and transparency as key takeaways, it arguably overstated their importance compared to the expert's broader, more nuanced points.
Claude also stood out for its ability to distil complex topics into digestible summaries. For instance, it accurately pulled out the gap between technical understanding and the ability to communicate with non-technical stakeholders, a theme with real implications for policy and innovation. It also surfaced crucial insights around ‘technical moats’ benefitting large tech companies, and how mechanisms like AI watermarking could foster public trust.
Gemini, while competent, often settled for a surface-level overview. It did identify themes like regulatory moats and the need to bridge expertise gaps, but without the clarity or framing that made Claude’s output so compelling. Overall, Claude delivered the most grounded, detailed, and human-readable summary of expert perspectives - something that clearly sets it apart in this context.
Actionable recommendations
Winner: Gemini
Consistent with how it performed in previous sections, ChatGPT’s actionable recommendations from the discussion were the most broad. Recommendations like “implement safeguards against misinformation” were too abstract to be genuinely helpful, and while it did suggest collaboration between industry leaders and regulators - a unique and worthwhile idea - it failed to provide actionable depth across most points. There was little sense that the suggestions were grounded in the specific conversation, and they certainly didn’t feel tailored to distinct audiences.
Gemini, on the other hand, delivered a no-nonsense list of actionable ideas. It prioritised clarity, with succinct, practical steps - particularly for communicators. For example, it highlighted the journalist’s recommendation to balance sensationalism with accuracy, a concrete takeaway that communicators could actually implement. Its suggestions felt intentional and applicable, especially compared to ChatGPT’s more generic phrasing.
Claude offered some of the most thoughtful and well-developed recommendations, even pulling in practical examples to illustrate how ideas like improving public understanding or reducing entry barriers for smaller players might work in practice. However, its biggest shortcoming was a lack of segmentation. Unlike Gemini and ChatGPT, Claude didn’t distinguish between recommendations for businesses, policymakers, or communicators and indeed most of its suggestions seemed geared toward business leaders. For communicators and policymakers, the guidance was less directly usable.
Notable Quotes
Winner: it’s a matter of opinion
Each of the chatbots pulled out completely different quotes from each of the three speakers; there was not a single quote which was pulled out by the same chatbot.
So, the tool that surfaced the most ‘notable’ quote was ultimately chosen based on a subjective (and human) judgment, made by someone who listened to the webinar (aka, the author of this blog post).
For Jasper Hamill, the most interesting quote was drawn out by Gemini:
"I think it is actually important to be sensational. It's important to talk in terms that are easily understandable by the populace."
For Susi O’Neill, Claude not only pulled out the most quotes (four, compared to Gemini’s two, and ChatGPT’s one), it also pulled out one of the most interesting:
"The further away you are from a typical Silicon Valley Tech bro, the less likely you are to embrace AI, and the more likely you are to see the negative consequences."
For Aled Lloyd Owen, an expert on regulation, ChatGPT drew out the most readable and engaging quote:
"AI isn’t something we can simply regulate in isolation. It needs the input of various voices, including those of ethicists, technologists, and the public, to create a responsible framework that ensures AI benefits everyone fairly."
In summary, each AI tool pulled out one of the most notable quotes depending on the speaker. This outcome, more than any other in this test, highlights the importance of human judgement when it comes to something as subjective as ‘notable quotes’.
It also reminds us that it’s important not to assume each tool has captured the entire summation of an hour-long discussion based on just one or two sentences. If you're after something specific, it's always worth following up with targeted prompts to guide your AI tool.
To summarise the summaries...
Claude emerged as the most thoughtful and insightful when it came to expert opinions, offering nuance, detail, and context that elevated its analysis. Gemini excelled in providing clear, actionable recommendations, especially for communicators, and consistently avoided vague generalities. ChatGPT, while occasionally surfacing unique ideas, tended to be the most generic across the board, often lacking specificity and relevance.
Most importantly, this exercise underlines that while AI can be a powerful partner in distilling complex discussions, it is not infallible. Each tool interpreted the same content differently, and not one captured every key insight or quote. Human judgement still plays a vital role in evaluating what matters, especially when dealing with qualitative nuance, expert perspectives, and subtle context.