Skip to main content
This feature is currently in beta and requires enablement. Contact us on Slack to get access.

​
Policy Preview Overview (Growth Plan)

Policy Preview is a beta CL feature that benchmarks query performance for a candidate policy — a new or modified policy you’re testing — and compares it to your current production policy. Use this tool to test policy changes during development or enforce performance checks in CI pipelines.

​
Requirements

  • Growth plan subscription
  • Oso Cloud CLI, version >= 0.35.0
  • A candidate policy to compare against your deployed policy
  • A list of queries to benchmark

​
Quick Start

  1. Create a query manifest (e.g. queries.yaml)
A YAML file that defines queries to preview. See Query Manifest Reference for details. 
# Example
queries:
  - name: "Admins can read repos"
    query_type: authorize
    actor_type: User
    actor_id: alice
    action: read
    resource_type: Repository
    resource_id: backend-repo

  - name: "List of repos Alice can read"
    query_type: list
    actor_type: User
    actor_id: alice
    action: read
    resource_type: Repository
  1. Create a candidate policy with your proposed changes You can use a single .polar file (e.g. candidate.polar) or multiple .polar files (e.g. policies/*.polar).
  2. Run the preview 
oso-cloud experimental policy-preview --queries queries.yaml policies/*.polar
This command runs all queries against both your production and candidate policies, then displays a side-by-side performance comparison.
  1. Define failure thresholds
You can configure thresholds that mark queries as failed when exceeded:
FlagDescription
-f,
--fail-on-regression-percent		Fail if any query is slower by this percentage
(e.g., 20 for 20%)
-i,
--ignore-changes-under-ms		Ignore regressions smaller than this threshold in milliseconds
(e.g., 10)
--max-duration-msFail if any query exceeds this absolute duration in milliseconds
(e.g., 750)
--max-regression-msFail if any query slows down by more than this many milliseconds
(e.g., 200)
Queries that exceed these thresholds appear as failed in the terminal output and cause the command to exit with code 1. Note: Individual queries that take longer than 1 second to evaluate will time out. A query fails if:
  • It times out with the candidate policy but not the current one, or
  • It times out with both policies.

​
Query Manifest Reference

The manifest file defines the queries to execute during policy preview. It supports the following query types: The query structure is similar to the HTTP API, with two additional required fields: name and query_type.

​
Manifest Structure

queries:
  - name: "Query name"
    query_type: <type>
    # Additional fields depend on query type

​
Query Types

​
Authorize Query

- name: <String> (must be unique)
  query_type: authorize
  actor_type: <String>
  actor_id: <String>
  action: <String>
  resource_type: <String>
  resource_id: <String>

​
List Query

- name: <String> (unique)
  query_type: list
  actor_type: <String>
  actor_id: <String>
  action: <String>
  resource_type: <String>

​
Actions Query

- name: <String> (unique)
  query_type: actions
  actor_type: <String>
  actor_id: <String>
  resource_type: <String>
  resource_id: <String>

​
Evaluate Query

- name: <String>
  query_type: evaluate_query
  predicate: [<String>]
  calls:
    - [<String>]
  constraints:
    <String>:
      type: <String>
      ids: [<String>]
    <String>:
      type: <String>
Required fields:
  • predicate: Array where first element is predicate name, the rest are variable names
  • constraints: Map of variable names to constraint definitions
    • type: Entity type for the variable
    • ids (optional): List of IDs to test
  • All variables in predicate or calls must be defined in constraints
Optional fields:
  • calls: Array of predicate calls formatted like predicate

​
Output Format

Policy Preview outputs results in a formatted table comparing baseline and candidate performance, showing regressions, improvements, and failures: 
Policy Performance Preview
╭──────────────────────────────────────┬────────────────┬───────────────┬─────────┬────────┬───────────────────────────────────────────────────────────────────────╮
│ Query                                ┆   Baseline p50 ┆ Candidate p50 ┆  Change ┆ Status ┆ Reason                                                                │
╞══════════════════════════════════════╪════════════════╪═══════════════╪═════════╪════════╪═══════════════════════════════════════════════════════════════════════╡
│ Admin list repos                     ┆ cold:     45ms ┆          52ms ┆  +15.6% ┆   ❌   ┆ Regressed 15.6% (limit: 10%)                                          │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│                                      ┆ warm:     12ms ┆          11ms ┆   -8.3% ┆   ✅   ┆                                                                       │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│                                      ┆                ┆               ┆         ┆        ┆                                                                       │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Check repo permissions               ┆ cold:    233ms ┆         751ms ┆ +222.3% ┆   ❌   ┆ Exceeded max duration limit of 500ms; Regressed +518ms (limit: +30ms) │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│                                      ┆ warm:    233ms ┆         240ms ┆   +3.0% ┆   ✅   ┆                                                                       │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│                                      ┆                ┆               ┆         ┆        ┆                                                                       │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Baseline vs. candidate timeouts      ┆ cold:  TIMEOUT ┆         150ms ┆   -INF% ┆   ✅   ┆                                                                       │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│                                      ┆ warm:    233ms ┆       TIMEOUT ┆   +INF% ┆   ❌   ┆ Candidate timed out                                                   │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│                                      ┆                ┆               ┆         ┆        ┆                                                                       │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Both timeout                         ┆ cold:  TIMEOUT ┆       TIMEOUT ┆      -- ┆   ❌   ┆ Both timed out                                                        │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│                                      ┆ warm:  TIMEOUT ┆       TIMEOUT ┆      -- ┆   ❌   ┆ Both timed out                                                        │
╰──────────────────────────────────────┴────────────────┴───────────────┴─────────┴────────┴───────────────────────────────────────────────────────────────────────╯
Summary: 0 passed, 4 regressed
❌ Policy preview failed regression checks

​
Best Practices

​
Create Representative Query Sets

Include queries that represent:
  • Common authorization paths
  • Edge cases
  • Performance-critical queries
  • Recently changed policy rules

​
Troubleshooting

​
”Feature not enabled” error

Policy Preview must be enabled. Contact us on Slack to gain access.

​
YAML parsing errors

Check your manifest structure. Common issues include:
  • Missing required fields
  • Incorrect indentation
  • Unknown fields

​
Limitations

  • Context facts are not currently supported in query manifests