Reading your conversations with ChatGPT. Targeting authentication.
Refocusing the debate about AI security. Weaponising open source intelligence. And a nine-year-old, his dad's company credit card, and $118,000.
Readers
Conversations with ChatGPT may not be as private as some users think, according to leaked internal documents obtained by 404 Media. According to the material, hundreds of human contractors are reviewing the AI’s responses to help improve its behaviour, and that includes conversation history and summaries of information about users. OpenAI says identifying information is filtered before conversations reach reviewers, but acknowledges that the system isn’t perfect and sensitive details may get through. Anthropic told 404 Media that it undertakes similar reviews.
None of this is intended to be alarmist - merely to highlight how important it is to be aware of how AI works. Both OpenAI and Anthropic provide straightforward ways to prevent ordinary chats being used for model improvement in this way. ChatGPT users can switch off “Improve the model for everyone” under Data Controls, while Temporary Chats aren’t used for training and business products are excluded by default. Anthropic has a similar control under Privacy. But 404 Media’s report reinforces something we’ve always said: if information must remain genuinely confidential, it doesn’t belong online.
Threats
Authentication: Extortion gangs are targeting organisations with sophisticated phishing attacks that begin by impersonating IT help desks and telling employees that they need to update an authentication method to avoid losing access to corporate systems. If employees take the bait, they end up on a site designed to resemble Microsoft login pages. No matter how credible, never trust calls or messages related to authentication. Microsoft
Complaining: A reminder that posting public complaints on social media is an invitation to be scammed. Check Point reveals the scale of criminal operations to monitor social media, impersonate customer support accounts, and contact people seeking help. Airlines provide a particularly rich hunting ground, partly because passengers may need immediate solutions to urgent problems. Social media can be a very effective mechanism to resolve issues, but don’t post in public.
Hijack: On a similar theme, attackers took over HBO Max’s official Reddit account and used it to launch 108 malicious ads over roughly 48 hours. Some promoted a fake HBO Max macOS app; others advertised AI and developer tools. Victims were sent to convincing sites that instructed them to paste commands into Terminal or PowerShell, with the aim of ultimately installing various malicious software. HudsonRock
Secrecy: Criminals have been using fake NDAs to make fraud more convincing. In one attempted €627,000 theft, the NDA instructed an employee to discuss a supposed confidential acquisition only via WhatsApp and personal email. Unhappily for the scammers, the intended victim wasn’t fooled but played along - allowing investigators to detail their entire plan. Gen Digital
Billing: ChatGPT users are being targeted with fake subscription emails claiming their payment method needs updating within 48 hours. The emails use OpenAI branding and link to a convincing copy of the ChatGPT login page, where any credentials entered are sent to the attackers. Cofense
AI
Coherence: Amid talk of a new silicon species and human extinction, cybersecurity professionals are suggesting it might be worth focusing on some security basics. In other words, before inventing an entirely new discipline of “AI safety”, there is a lot that could be learned from conventional cybersecurity: assume systems will be attacked, restrict privileges, monitor behaviour, isolate failures and have mechanisms for responding when safeguards fail. Not so headline grabbing as potential catastrophe, but arguably more practical.
Claude: In another step towards AI chatbots becoming general-purpose digital workers, Anthropic is folding its Cowork agent into the main Claude chat interface. Rather than choosing between simply asking a question and handing over a task, Claude will now decide for itself whether a request needs a quick answer or a longer, multi-step job. It’s all part of a wider “superapp” strategy (mirrored by OpenAI) that would fold disparate tools like chat, documents, slides, and spreadsheets into a single interface - ideally, one you never leave. That might be convenient, but it also has enormous implications for security. As we keep saying, the time to think about what an agent can access is before you hand it the keys, not afterwards.
Training: Apple has reversed its stance on using interactions with Siri to improve its AI models. Beginning with iOS 27, users are asked whether they want to share their interactions. Choosing “Not Now” declines the request without disabling the assistant. For the moment, it’s something of a moot point because you have to join a waitlist to access Siri AI.
Horror: AI agents are flooding social media platforms and writers with spam messages in an attempt to gain traction for a startup. iLands describes itself as “a shared network where autonomous AI Agents and humans create, work, exchange value, build relationships, and live with consequences that carry forward.” Showering the Internet with spam doesn’t feel like an optimal way to begin. 404 Media
Romance: A network of dating apps used thousands of AI-generated women to persuade users to pay for conversations. Most chats involved no human at all, although paid workers occasionally appeared on video to make the deception convincing. Anthropic identified 4,700 fake personas interacting with 25,000 people in just two weeks. In one case, a researcher received a video call from “Jennifer”, who told him his voice was better than expected. This despite the fact his microphone wasn’t even turned on.
In brief
OSINT: An investigation by Israeli media reveals how information exposed on social media can be turned into a surveillance tool. Leaked documents obtained by The Marker show Israeli firm CyberGlobes helping a range of authoritarian regimes to track down dissidents, activists and political rivals. In Indonesia, it helped officials identify LGBTQ people and map the groups they belonged to. CyberGlobes disputes the report.
Tailored: Iran is using highly personalised lures to infect targets with spyware, including sending one victim what appeared to be an MRI scan of a herniated disc. British, US and Dutch security agencies say dissidents, activists and journalists are among the targets. It’s worth noting that the malicious file is only sent after the attackers have established trust, often by communicating on WhatsApp or Telegram and impersonating a known contact or technical support.
Cameras: Controversy continues to surround Flock cameras, with Boston abandoning them because of privacy concerns and reports showing the devices are riddled with security vulnerabilities. Boston’s city government dumped the number-plate readers after the company shared data nationwide in violation of a contract provision. Meanwhile, hackers ripped a camera from its mounting and published part of its filesystem. Early analysis shows it running an unsupported version of Android which hadn’t received any security updates for the past eight years.
Passkeys: The UK government is rolling out passkeys across its GOV.UK One Login, giving more than 23 million people the chance to ditch passwords - and improve their security. The move means users will be able to sign in with a fingerprint, Face ID, or device PIN instead of entering a password and waiting for a two-factor authentication code. Good!
Ouch: A nine-year-old spent $118,000 on YouTube adverts for his Minecraft and Roblox gameplay videos - on his dad’s company credit card. The father, who identifies himself only as Dave, describes the incident in a mournful video, in which he says he only found out what had happened when he was called into a meeting by his managers. He may now lose his job. Cautionary tales don’t come much more cautionary than this. Dave is a marketing professional, who seems to have had a somewhat carefree approach to his company card and says he didn’t realise the possible impact of saving the card information online. Mind you ‘Mighty Mike Plays’ now has 128,000 subscribers on YouTube, so maybe it will start earning back some of those dollars. And, of course, we’re assuming this isn’t just a clever marketing ploy by Mighty Mike and his dad…
Updates
Apple: Major updates across Apple’s products, including macOS 27 and iOS/iPadOS 27. So far, there are no reports of significant issues with the updates, though do be aware that macOS 27 takes up a lot of space and there are some isolated examples of app incompatibility (e.g. Cisco Duo). If you don’t want to install the new versions straight away, there are updates for the previous OS 26 generation. Whichever you choose, it’s worth installing immediately because they contain fixes for hundreds of security vulnerabilities.
Microsoft: This month’s updates have resulted in several undesired consequences including audio problems on some Windows PCs and copy/paste failures in Excel. The audio issue has been partially fixed. Microsoft has addressed the issue in Excel 2016 but is still working on solutions for other versions.
TP-Link: Vulnerabilities in Tapo C200 cameras could let hackers spy inside homes. The issues have been fixed and underline why it’s essential to ensure smart devices like cameras, plugs, and lights are kept up to date.
Cisco: Critical updates for Secure Email Gateway and Identity Services Engine.
Oracle: September edition of Critical Security Patch Update includes patches for 673 security vulnerabilities.
Google: September security updates address 110 issues affecting Pixel devices, including a “zero day” (previously unknown) vulnerability that is being actively exploited.