WardrikPolicy.Budget (wardrik_policy v0.1.0-rc.1)

Copy Markdown

Caps what an agent spends on model calls, in dollars, tokens or both.

iex> policy = WardrikPolicy.new!(budget: [usd: 5.00, per: :day])
iex> WardrikPolicy.Budget.check(policy.budget, %{usd: 4.10})
:allow
iex> WardrikPolicy.Budget.check(policy.budget, %{usd: 5.02})
{:ask, "This agent has spent $5.02 of $5.00 today. Approve to allow $5.00 more."}

Call check/3 before each model call, with what the earlier calls spent. The call that crosses the limit still runs. The next one asks. grant/2 records an approval and allows another grant's worth:

iex> policy = WardrikPolicy.new!(budget: [tokens: 200_000, grant: [tokens: 50_000]])
iex> spent = WardrikPolicy.Budget.grant(policy.budget, %{tokens: 201_000})
iex> WardrikPolicy.Budget.check(policy.budget, spent)
:allow
iex> WardrikPolicy.Budget.check(policy.budget, %{spent | tokens: 251_000})
{:ask, "This agent has used 251,000 of 251,000 tokens today. Approve to allow 50,000 more."}

cost/2 prices a call that reports no cost at the budget's price, in dollars per million tokens:

iex> policy = WardrikPolicy.new!(budget: [usd: 5.00, price: [input: 0.60, output: 2.20]])
iex> WardrikPolicy.Budget.cost(policy.budget, %{input_tokens: 10_000, output_tokens: 2_000})
0.0104

What was spent is a map with :usd, :tokens and :granted. A missing key counts as zero. Reset it when period/2 returns a new period. WardrikPolicy.Middleware does all of this for you.

Options

  • :usd - the limit in US dollars. No dollar limit unless set.
  • :tokens - the limit in tokens, input and output together. No token limit unless set.
  • :per - :turn, :day, :week, :month or :total. :day unless set.
  • :timezone - where a day, week or month begins, as "Europe/Berlin". "Etc/UTC" unless set. Weeks begin on Monday.
  • :grant - how much more an approval allows, as [usd: 2.00]. Another limit's worth unless set.
  • :price - dollars per million tokens, as [input: 0.60, output: 2.20]. Used for calls that report no cost.
  • :on_exhausted - :ask a person, or :deny. :ask unless set.

Summary

Types

period()
spent()
t()

A budget. See the module docs for its fields.

Functions

check(budget, spent, opts \\ [])

Checks what was spent against the budget.

cost(budget, usage)

Returns what a call cost in dollars.

decode(fields)

Reads a budget from a map with string keys, such as one parsed from YAML.

grant(budget, spent)

Records an approval. Each limit already reached allows another :grant's worth on top of what was spent.

period(budget, now)

Returns the period now falls in. Reset what was spent when it changes.

to_json(budget)

Encodes a budget as the map decode/1 reads. Leaves out what the budget doesn't set.

Types

period()

@type period() :: :turn | :total | {:day | :week | :month, Date.t()}

spent()

@type spent() :: %{
  optional(:usd) => number(),
  optional(:tokens) => non_neg_integer(),
  optional(:granted) => %{
    optional(:usd) => number(),
    optional(:tokens) => integer()
  },
  optional(atom()) => term()
}

t()

@type t() :: %WardrikPolicy.Budget{
  grant: keyword(number()),
  on_exhausted: :ask | :deny,
  per: :turn | :day | :week | :month | :total,
  price: [input: number(), output: number()] | nil,
  timezone: String.t(),
  tokens: pos_integer() | nil,
  usd: number() | nil
}

A budget. See the module docs for its fields.

Functions

check(budget, spent, opts \\ [])

@spec check(t(), spent(), keyword()) :: WardrikPolicy.decision()

Checks what was spent against the budget.

Returns :allow while under every limit. Otherwise returns {:ask, prompt} or {:deny, message}, as :on_exhausted says.

iex> policy = WardrikPolicy.new!(budget: [usd: 5.00, on_exhausted: :deny])
iex> WardrikPolicy.Budget.check(policy.budget, %{usd: 5.02}, spender: "acme's agents have")
{:deny, "acme's agents have spent $5.02 of $5.00 today."}

Options

  • :spender - who spent it, at the start of the message. "This agent has" unless set.

cost(budget, usage)

@spec cost(t(), map()) :: number() | nil

Returns what a call cost in dollars.

Uses the call's own :usd when it has one. Otherwise prices its :input_tokens and :output_tokens at the budget's :price. Returns nil when the budget has no price.

decode(fields)

@spec decode(map()) :: {:ok, t()} | {:error, String.t()}

Reads a budget from a map with string keys, such as one parsed from YAML.

grant(budget, spent)

@spec grant(t(), spent()) :: spent()

Records an approval. Each limit already reached allows another :grant's worth on top of what was spent.

period(budget, now)

@spec period(t(), DateTime.t()) :: period()

Returns the period now falls in. Reset what was spent when it changes.

Returns :turn for a :turn budget, so reset it at each turn yourself. Raises ArgumentError if :timezone isn't a time zone.

to_json(budget)

@spec to_json(t()) :: map()

Encodes a budget as the map decode/1 reads. Leaves out what the budget doesn't set.