Chat APIs structure input as roles: system (persistent instructions/persona), user (the human turn), assistant (model turns), and increasingly developer (app-owner rules ranked above user).
How it works
The model operates by assigning different weights to the roles defined in the input structure: system, user, assistant, and developer. The system role contains persistent instructions that shape the model's behavior, while the developer role allows the app owner to impose rules that take precedence over user inputs. This hierarchy ensures that even if a user tries to manipulate the conversation, the core guidelines set by the system and developer remain intact. All roles share a single context window, meaning they can influence the model's responses, but the priority is clearly defined to maintain control.
For example, if a user asks the model to perform an action that contradicts the established system instructions, the model will prioritize those instructions. If the system role states that the model should not provide personal data, it will override a user request for that information. This mechanism prevents users from bypassing safety protocols and ensures that the model adheres to the intended guidelines, thereby maintaining a safe interaction environment.
the model's ability to distinguish between these roles allows it to manage context effectively. When it processes input, it recognizes the role of each piece of text and adjusts its response accordingly. This nuanced understanding helps the model to generate outputs that align with the developer's objectives while still addressing user queries, creating a balanced interaction that respects both user needs and system integrity.
- Use system for durable rules.
- Use developer for app-owner policies.
- Use user for actual requests.
- Never include secrets in prompts.
- Maintain context for all roles.
The software-engineering analogue
In software engineering, we often deal with privilege levels, such as distinguishing between root configuration settings and user input. The system and developer roles can be compared to these higher privilege levels, where certain configurations dictate the behavior of the application. However, the key difference here is that the boundary between these roles is not enforced by strict permission checks. Instead, it's a soft boundary shaped by the model's training, meaning that a clever user can still manipulate inputs to achieve unintended results.
Why not the obvious alternative
A common initial approach might be to concatenate all inputs into a single blob, treating them equally. This method loses the crucial priority signal that differentiates between user requests and system instructions. Without this separation, the model becomes vulnerable to prompt injection attacks, where a user could craft inputs designed to exploit weaknesses in the system. This could lead to situations where the model inadvertently follows harmful or unintended instructions.
By maintaining a clear distinction between roles, the model can better defend against conflicting instructions. When user text contradicts the policies set by the system or developer, the model knows which guidelines to prioritize. This layered approach not only enhances security but also ensures that the model operates within the intended parameters, delivering reliable and safe interactions.
When to use which
When designing your application, use the system and developer roles for establishing durable rules, defining the model's persona, and outlining the output contract. These roles are essential for maintaining safety and ensuring that the model adheres to your policies. On the other hand, the user role should be reserved for capturing actual requests and untrusted data. It's crucial to avoid placing sensitive information or hard trust boundaries directly in the prompt; instead, enforce these constraints through your application code to ensure security.
In production: when it breaks
Why it happens. When the system role is ambiguous or improperly set, the model struggles to generate contextually relevant responses. This confusion leads to misunderstandings in conversation flow, resulting in lower user satisfaction as users receive irrelevant or off-topic replies.
- Check the configuration of the system role in the API settings.
- Log the last 100 interactions to identify patterns in user dissatisfaction.
- A/B test with a clearly defined system role versus an ambiguous one.
What it delivers
This structured approach provides a clean, defensible separation between your application's rules and user input. It allows for safe interactions while maintaining control over the model's behavior, ultimately leading to a more reliable and secure user experience.