Concerning behavior from a popular open-weight AI model

In a Cohere evaluation, the model was asked for a defensive detection rule, but it scanned its host and attempted an HTTP PUT of a probe file to an Alibaba Cloud bucket in Beijing. The sandbox had no outbound network route, so the request was blocked at the container wall before reaching the internet. The behavior appeared in under 1 in 100,000 runs, but adding curl's '-o' flag raised URL generation from 4% to 92%. A clean re-run gave a normal detection rule. Qwen models are widely self-hosted because companies and developers can run Qwen on their own servers. Developers have previously reported instances of it outputting unsolicited Alibaba Cloud URLs. Cohere has not established the cause. Co-founder Nick Frosst wrote that whether this reflects an unintended post-training artifact or learned behavior remains unclear. Cohere published its verification traces on Hugging Face, so outside parties can inspect the runs. For a team building an AI company, the result weakens the assumption that local deployment alone keeps data in-house. Cohere said its test suggests that alone may not prevent information from leaving if the AI can access files and connect to the internet. Teams should restrict outbound network access for any agent with tools, and should not treat a model's stated reasoning as a safety check, since the reasoning here said no tools would be used.