When building systems that use language models, it's tempting to solve everything from the prompt. But when it comes to business rules, repeatable decisions, or audit requirements, the right approach is to separate responsibilities: the text is handled by the LLM, but the rules should reside outside.

1. Separation of Concerns: Text vs. Rules

LLMs are excellent language generators, but they are not designed to reliably, explainably, or reproducibly apply business logic. Therefore, decisions that affect the behavior of a system should:

• Use structured data as the source of truth.
• Be executed by deterministic code (outside the model).
• Be traceable, testable, and versioned.

2. Blackboard/State Design

The state is a persistent structure, in JSON format or equivalent, that reflects the current context of the user or session:

• Declared preferences.
• Control flags.
• Previous results.

It must be accessible by tools and the LLM (if relevant for language generation), but always maintaining an explicit contract.

The state is consulted, not reinvented.

3. Deterministic Algorithms as Tools

State-based decisions should be implemented as external tools with clear inputs and outputs. Example:

{
  "input": {
    "prefs": {"mode": "safe"},
    "question": "¿Puede el usuario acceder a X?"
  },
  "output": {
    "decision": false,
    "reason_code": "PREF_SAFE_MODE_RESTRICTS"
  }
}

This allows:

• Encapsulating complex logic.
• Validating with tests.
• Justifying decisions with codes and traces.

4. Versioning and Traceability

Every state must have a schema version. Changes should be recorded with:

• Timestamp.
• Change identifier.
• Author (if applicable).

This facilitates migrations, debugging, and support for multiple versions.

5. Validation and Post-processing

When saving or using the state:

• Validate against a versioned JSON schema.
• Apply normalization or cleaning if necessary.
• Log changes in a structured way.

Example Table: State Structure

Field	Type	Source	Version	Last Update
userPrefs.mode	String	User	v1.2	2025-08-28
flags.beta	Boolean	System	v1.0	2025-08-25
history	Array	Interactions	v1.3	2025-08-27

Technical Checklist

• Versioned and validated schema.
• Migrations between versions.
• Unit tests for tools.
• Justifiable decision codes.

Frequently Asked Questions

• How much to put in the state and how much to leave in the prompt?

  • Anything that is structured, persistent, or auditable should go into the state. The prompt is only for communication, not for storing logic.