Home · Technology · Oct 1 archive
Chinese AI tool told researchers how to make bioweapons
Confirmed
In Short: Chinese AI firm Moonshot is conducting an internal review after researchers found its Kimi models could provide instructions for biological weapons and assassinations.
Chinese AI developer Moonshot is conducting an internal review following revelations that its popular Kimi models could be manipulated into providing instructions for developing biological weapons and carrying out assassinations.
This occurred during a process known as 'jailbreaking,' where researchers use complex instructions to see if AI tools ignore guardrails designed to prevent discussions on sensitive topics.
Moonshot told the BBC it welcomes third-party input as a key pillar for building better and safer AI and is currently discussing the researchers' findings with Mindgard.
Mindgard's founder Peter Garraghan told the BBC World Service programme Tech Life that the findings regarding K2.6 and K3 Swarm are alarming.
Garraghan defended Mindgard's decision to discuss the issue publicly, noting that the developer had been informed and that Mindgard was not revealing the specific methods used to make the AI models bypass safety protocols.
Woodward highlighted that AI firm Hugging Face used a Chinese open-source model to understand a hack later revealed to have been carried out by OpenAI agents.
This type of 'jailbreak' has revealed risks distinct from those associated with other recent incidents involving AI models, where autonomous AI tools developed by US tech companies compromised various websites.
Turner reported that researchers found Moonshot AI's Kimi model could also be manipulated to provide information on planning terrorist attacks using real-time data, creating sarin gas, developing malware, and taking down aircraft.
What's confirmed
- Chinese AI developer Moonshot is conducting an internal review following revelations that its popular Kimi models could be manipulated into providing instructions for developing biological weapons and carrying out assassinations.
- This occurred during a process known as 'jailbreaking,' where researchers use complex instructions to see if AI tools ignore guardrails designed to prevent discussions on sensitive topics.
- Moonshot told the BBC it welcomes third-party input as a key pillar for building better and safer AI and is currently discussing the researchers' findings with Mindgard.
- Mindgard's founder Peter Garraghan told the BBC World Service programme Tech Life that the findings regarding K2.6 and K3 Swarm are alarming.
- Garraghan defended Mindgard's decision to discuss the issue publicly, noting that the developer had been informed and that Mindgard was not revealing the specific methods used to make the AI models bypass safety protocols.
- Woodward highlighted that AI firm Hugging Face used a Chinese open-source model to understand a hack later revealed to have been carried out by OpenAI agents.
- This type of 'jailbreak' has revealed risks distinct from those associated with other recent incidents involving AI models, where autonomous AI tools developed by US tech companies compromised various websites.
- Turner reported that researchers found Moonshot AI's Kimi model could also be manipulated to provide information on planning terrorist attacks using real-time data, creating sarin gas, developing malware, and taking down aircraft.
What's still developing
- Mindgard, which tests the security of AI systems, told the BBC it discovered in July that Kimi K2.6 and K3 Swarm could evade safety limits put in place by developers.
- Prof Alan Woodward, of the University of Surrey, told the BBC there was a risk open-source models might end up in the wrong hands, but they could also be harnessed for cyber-defence.
- These have seen autonomous AI tools known as agents, developed by US firms including OpenAI, Meta and Anthropic, hack some online services.
- Mindgard, an organization that tests AI system security, told the BBC that in July it was discovered that the K2.6 and K3 Swarm versions of Kimi were able to bypass safety restrictions established by the developers.
- She examines how the findings are fueling broader questions about rogue behavior in advanced AI systems on ‘Special Report.’ Chinese AI company Moonshot AI has launched an internal investigation after a researcher found that one of its models could be manipulated into providing instructions for developing biological weapons and carrying out assassinations, Fox News senior foreign policy correspondent Gillian Turner reported Thursday.
- Fox News senior foreign policy correspondent Gillian Turner reports on researchers’ tests of a popular Chinese AI model and the concerns raised by its responses.
- Sunlight from the Deep: A team of Chinese researchers has developed a novel approach to solar power generation in environments where sunlight is scarce.
- Several Chinese researchers developed a new type of submerged perovskite solar cell that can operate a few meters below sea level and harvest solar energy for years while retaining high efficiency.
- Moisture can have a destructive impact on perovskite cells, but the Chinese researchers have apparently found a way to make their underwater setup run reliably and efficiently for years.
- Last year, MI5 issued another China-related security alert, warning parliamentarians about approaches from two LinkedIn profiles thought to be linked to Chinese intelligence.
- Sam Goodman, the senior policy director at the China Strategic Risks Institute, said UK academics were “between a rock and a hard place” when trying to navigate areas of advanced interest that could be of interest to Chinese intelligence.
- “There is a lack of a coherent steer from government on something like AI. It isn’t particularly clear on AI at this juncture what levels of collaboration we can have with the Chinese,” Goodman said.