Skip to main content
Safety and security should be treated as product requirements, not as a paragraph added after the feature ships.

​
13.1 Threat modeling

For each significant feature, identify assets, trust boundaries, potential attackers, likely failure paths, and mitigations. The analysis should include ordinary mistakes as well as deliberate abuse. Relevant assets may include account credentials, API keys, private files, model endpoints, tool permissions, billing records, training datasets, and physical devices.

​
13.2 Defense in depth

No single safeguard is enough. A robust design combines authentication, authorization, input validation, isolation, rate limits, monitoring, incident response, and recovery procedures. Model-level refusals can contribute to safety, but should not be the only mechanism preventing an unauthorized operation.

​
13.3 Dangerous or irreversible actions

Sending communications, publishing content, deleting files, changing permissions, spending money, or controlling physical hardware may require explicit confirmation. The confirmation should describe the action and its consequences clearly enough for the user to make an informed decision.

​
13.4 Cancellation and emergency controls

Cancellation should propagate through the orchestration system and tool runtime. A user should not be told that a task has stopped while a background worker continues executing it. The system should track cancellation state and handle operations that cannot be interrupted instantly. For future robotics, emergency stopping and safe-state behavior require independent engineering. A software kill switch is not a substitute for a hardware emergency stop, physical safeguards, and validated control systems.

​
13.5 Security testing

Testing should include dependency scanning, secret detection, access-control tests, API abuse tests, prompt-injection scenarios, connector-permission tests, and recovery exercises. High-impact changes should receive review proportional to their risk.

​
13.6 Incident response

A documented incident process should define severity levels, responsible responders, containment steps, evidence preservation, communication responsibilities, and post-incident review. Incidents should lead to corrective action and verified improvements, not merely a status post.

​
13.7 Responsible release

Before launching a major capability, Orbit should document its intended use, known limits, relevant evaluations, user controls, support path, and rollback mechanism. Capabilities with significant consequences should receive stronger review and more restricted initial access.