WardrikPolicy.Judge (wardrik_policy v0.1.0-rc.1)

Copy Markdown

Asks a model yes-or-no questions about each tool call, and decides by the probability of yes.

iex> policy =
...>   WardrikPolicy.new!(
...>     judge: [
...>       questions: [
...>         [
...>           question: "Does this call send money outside the company?",
...>           tools: ["books/*"],
...>           ask: 0.5,
...>           deny: 0.9
...>         ]
...>       ]
...>     ]
...>   )
iex> evaluate = fn _model, _state, _questions -> {:ok, [0.72], %{}} end
iex> WardrikPolicy.decide(policy, "books__pay_bill", evaluate: evaluate)
{:ask, "Judged 72% likely: Does this call send money outside the company?"}

Options

  • :model - the model that answers. "typesafe:jev-latest" unless set.
  • :questions - a list of questions, each with the keys below.

Each question takes:

  • :question - required. What the model is asked about the call.
  • :tools - tool names it applies to, as ["books/*"], without arguments. Every tool unless set.
  • :deny - refuse the call at or above this probability.
  • :ask - ask a person at or above this probability.
  • :allow - allow the call below this probability.
  • :message - the prompt or refusal to show in place of the question.

Each threshold is a probability from 0 to 1. A question needs at least one of them. decode/1 returns an error otherwise.

Evaluating

A call's questions go to the model in one request, with the tool's name and arguments. Pass a function as the :evaluate option of WardrikPolicy.decide/3 to make that request. See evaluate/0. WardrikPolicy.Middleware uses WardrikCore.evaluate/4. What the request used is returned in the decision's measurements.

How it combines with rules

  • A deny beats an ask, and an ask beats an allow, across questions and rules.
  • A judge's allow only decides calls no rule names. It never overrides a rule's ask or deny.
  • A judge's allow is ignored once the run has read content from outside.
  • If the model can't answer, the call asks a person, as long as some question could ask or deny.

The call's arguments may be written to fool the judge. Keep exact checks in rules, and test the thresholds on real calls before trusting the judge's allow.

Summary

Types

evaluate()

Asks model each question about state. Returns each question's probability of yes, in order, and what the request used.

question()
t()

Functions

decide(judge, name, arguments, evaluate)

Decides a call to the tool name by the judge alone.

decode(fields)

Reads a judge from a map with string keys, such as one parsed from YAML.

to_json(judge)

Encodes a judge as the map decode/1 reads.

Types

evaluate()

@type evaluate() :: (model :: String.t(), state :: map(), questions :: [String.t()] ->
                 {:ok, [float()], measurements :: map()} | {:error, term()})

Asks model each question about state. Returns each question's probability of yes, in order, and what the request used.

question()

@type question() :: %{
  :question => String.t(),
  optional(:tools) => [String.t()],
  optional(:deny) => float(),
  optional(:ask) => float(),
  optional(:allow) => float(),
  optional(:message) => String.t()
}

t()

@type t() :: %WardrikPolicy.Judge{
  model: String.t(),
  questions: [question() | keyword()]
}

Functions

decide(judge, name, arguments, evaluate)

@spec decide(t(), String.t(), map(), evaluate() | nil) ::
  WardrikPolicy.decision() | :pass

Decides a call to the tool name by the judge alone.

Returns :allow, {:ask, prompt} or {:deny, message}. Returns :pass when no question applies or none crosses its threshold.

decode(fields)

@spec decode(map()) :: {:ok, t()} | {:error, String.t()}

Reads a judge from a map with string keys, such as one parsed from YAML.

to_json(judge)

@spec to_json(t()) :: map()

Encodes a judge as the map decode/1 reads.