Home · Technology · Sep 23 archive
Meta's AI Assistant Muse Hit by Zero-Day Vulnerability
Confirmed
In Short: Meta's AI assistant Muse, designed to be the brains behind its upcoming smart glasses, has been hit by a serious zero-day vulnerability, raising significant security concerns.
Meta's AI assistant Muse, designed to be the brains behind its upcoming smart glasses, has been hit by a serious zero-day vulnerability, raising significant security concerns.
According to Jane Manchun Wong, a renowned reverse engineer, Meta is testing an option to set Muse as the default assistant for its connected eyewear, indicating the company's plans to integrate the AI deeply into its ecosystem.
The vulnerability, disclosed by security researcher Kevin Beaumont, allows any locally installed app or executed code to gain access to the token that authenticates users to their Muse account.
This zero-day flaw was discovered just two weeks after Muse's launch on September 8, and it was patched shortly after its discovery.
The flaw involved Muse's dictation feature, which sends audio to Meta's servers for transcription. An attacker with code running on the user's Mac could redirect this traffic to a server they control, gaining access to the user's audio and Muse account authentication token.
This could potentially allow attackers to hijack the assistant and exploit permissions users have already granted to Muse.
Meta founder Mark Zuckerberg had previously emphasized the security and privacy of Muse, stating it was built from the ground up with these principles in mind.
Amazon also began blocking Muse from its site shortly after the vulnerability was disclosed, citing lack of prior notification and the assistant's inability to identify itself while shopping.
Meta's security team acknowledged the issue, describing it as a local privilege escalation attack that required malicious code already running on the user's machine.
Despite the low practical risk to users of the Muse Mac app, Meta issued a hotfix to address the issue, underscoring the importance of addressing such vulnerabilities promptly.
The vulnerability highlights the broader risks associated with AI assistants that require extensive access to user data and device features.
What this adds
This report adds details about the specific nature of the zero-day vulnerability and its potential impact, as well as Meta's response to the issue. It also highlights the broader security concerns surrounding AI assistants that have access to a wide range of user data and device features.
Background
Meta's new AI assistant, Muse, is drawing both praise and criticism for its capabilities and data collection practices.
What's confirmed
- Further raising questions, Amazon on Sunday began blocking Muse from its site.
- Meta has published two posts in as many weeks documenting the design decisions that went into ensuring an assistant with such extraordinary access to user data and resources is secure and private.
- “To me, the bar is infinitely higher in terms of the security of these apps. They don’t have to be perfect, but when you take a look at Muse, it’s like they didn’t, in my opinion, think about security, which is really worrisome,” Wardle said.
- Roughly 12 hours before Wardle disclosed the zero-day, Amazon started blocking people from using Muse to shop on the site.
- Meta founder and CEO Mark Zuckerberg has gone to great lengths to hype the security of its new AI assistant, Muse, claiming it is “built from the ground up for privacy and security.” A zero-day vulnerability that gives locally run apps and terminal commands complete control of the agent raises serious doubts.
- The assistant “books appointments, fills out forms and handles customer service,” “proactively takes tasks off your plate,” and can “make purchases, generate images, create documents, and connect with your favorite apps and services.” The macOS app (curiously, there’s no Windows version) also works with a user’s WhatsApp, email, calendar, and social media accounts.
- When a task requires a tool that doesn’t exist, Muse creates one on the fly.
- Of course, for Muse to do any of these things, users must first give it access to their accounts.
- This includes authenticating the assistant to each service and, because the app runs on macOS, giving it permissions to a broad range of operating system-restricted device resources, like writing files to disk, accessing the mic and camera, and monitoring location and calendars.
- The zero-day allows any app or terminal command to gain access to the token that authenticates users to their Muse account.
- Meta developers designed the assistant so that any locally installed app or executed code, regardless of the macOS permissions it has, can change a long list of undocumented settings.
What's still developing
- A separate leak shared by Wong on X shows Meta is also working on “Meta Pay for Muse,” developing a native wallet interface for Muse with Meta Pay support, alongside Stripe’s Link and Shop Pay.
- The leaked setup screen, titled “Enable Impacc as your glasses assistant?” (where “Impacc” likely refers to an internal codename for Muse), notes that Muse is in beta and will allow users to swap back to Meta AI whenever they prefer.
- Meta launched Muse on Sept. 8 as a personal AI agent that can handle tasks like sending emails, booking travel, shopping, and tracking goals.
- “We strive to be extremely transparent about privacy and security in Muse as we know this is important to maintain your trust,” Singleton wrote in a post on X explaining the issue.
- According to Singleton, the version of Muse that shipped included an internal setting that allowed developers to change the server endpoint used for the dictation feature, something he said was useful for debugging and development.
- The wearable best suited to actually carry a conversational AI assistant everywhere has been sitting in your ears for a decade.
- To do all that, users give Muse access to a wide range of apps, accounts, and device features, making any vulnerability that could hijack the agent particularly concerning.
- Most of them are fairly innocuous, such as controlling dark mode.
- It allowed processes to change the end point where transcription occurs.
- Attackers could have exploited this flaw by changing the location to their own end point.
- A prominent macOS security researcher has urged Mac users not to install Meta's new Muse AI assistant, publishing proof-of-concept code for what he described as a zero-day that can turn the app into a ready-made backdoor.