Four things a formula evaluator must do that eval() cannot
A spreadsheet formula is a program somebody else wrote, and sooner or later it has to run outside the spreadsheet, either in a service that recomputes a model every month or in a test that proves the balance sheet still balances. The shortest path in Python is eval() over the formula string with the line values as a namespace, and the usual objection is that this executes arbitrary code. That objection is correct. It is also the one everybody already knows, and the least interesting reason to write a parser instead.
Assume the best case against us. The formulas are authored in-house, reviewed before they ship, stored in a database nobody outside the team can write to, and never submitted by a user. The security argument is weak now. eval() still fails three other jobs a formula language has to do: refuse everything that is not arithmetic, so there is nothing to escape into even by accident; fail at construction rather than at the first number; and know the order the lines must be evaluated in before any value is supplied. A tokenizer and a recursive-descent parser give you all four. eval() gives you none of them, the security one included.
The constraint#
We build financial statements as data rather than as cells. In finmodel a statement is a tree of frozen dataclasses: a Block groups categories, an InputLine takes values per period, and a ComputedLine carries a formula that names other lines by id. Nesting is there so you can read the statement, and it creates no scope: ids are flat across the whole thing, so a formula in one block may name a line in another.
from finmodel import Block, ComputedLine, InputLine, Statement
pnl = Statement(
id="pnl",
label="Profit and loss",
children=[
Block(
id="revenue",
label="Revenue",
children=[
InputLine(id="product_sales", label="Product sales"),
InputLine(id="services", label="Services"),
ComputedLine(
id="total_revenue",
label="Total revenue",
formula="product_sales + services",
),
],
),
Block(
id="costs",
label="Costs",
children=[InputLine(id="cogs", label="Cost of goods sold")],
),
ComputedLine(
id="gross_profit",
label="Gross profit",
formula="total_revenue - cogs",
),
],
)
pnl.evaluation_order
# ('total_revenue', 'gross_profit')The twelve months of a year are that one spec evaluated twelve times, not twelve copies of it. That is what puts all the weight on construction. The numbers arrive many times over, the spec arrives once. Anything wrong with the model should be wrong before a single value exists, because once values exist you are no longer looking at the model, you are looking at a column of amounts.
Nothing to escape into#
What makes a formula language interesting is what it cannot express, rather than what it rejects. With eval(), total_revenue - cogs works, and so does every attribute access, call, subscript and comparison in the language, which is why hardening it turns into a list of names to strip from the namespace and dunders to blacklist. That list is never finished, because it is the wrong shape of defence.
Our tokenizer, _TOKEN_RE, recognises numbers, names, the four arithmetic operators and parentheses. Nothing else is a token. _Parser is a recursive descent over that stream and builds four node types: Number, Reference, UnaryOp, BinaryOp. There is no production for a call, an attribute, a subscript or a comparison, and a dot is not a token outside a number. So revenue.__class__ is not a formula that gets refused by a rule we remembered to write. It is not a formula. FormulaError names the position where the input stopped being one.
Every node answers the same two questions, which ids it reads and what it evaluates to in a given scope:
@dataclass(frozen=True)
class Reference:
"""A read of another line, by id."""
name: str
@property
def references(self) -> tuple[str, ...]:
return (self.name,)
def evaluate(self, scope: Scope) -> Amount:
try:
raw = scope[self.name]
except KeyError:
raise FormulaError(f"unbound reference {self.name!r}") from None
return to_amount(raw, self.name)Note to_amount. Values coming in from a loader are coerced to Decimal, and a value that is not a number raises FormulaError naming the line it came in on. Under eval() the same inputs would quietly become binary floats, and two strings would concatenate instead of adding. A money model that computes the right arithmetic on the wrong number type has not been saved by the fact that the formula parsed.
Construction is where a formula fails#
A ComputedLine parses its formula when it is constructed, and re-raises the parse failure as a SpecError that names the line, chained from the original FormulaError:
def __post_init__(self) -> None:
super().__post_init__()
if not self.formula.strip():
raise SpecError(f"computed line {self.id!r} must have a formula")
try:
expression = parse(self.formula)
except FormulaError as exc:
raise SpecError(f"formula of {self.id!r} is invalid: {exc}") from exc
object.__setattr__(self, "expression", expression)
object.__setattr__(self, "references", expression.references)The parsed tree is kept as line.expression and the ids it reads as line.references, so the formula string is parsed once in the life of the spec instead of once per period. Statement then validates the whole tree at construction, and refuses with a message that says which line is at fault:
| The spec says | The error reads |
|---|---|
| an id that is not an identifier | line id must be an identifier, got 'total revenue' |
| the same id twice | duplicate id 'services' |
| a computed line with no formula | computed line 'gross_profit' must have a formula |
| a reference to nothing | line 'gross_profit' references unknown line 'cgos' |
| a reference to a block | line 'x' references block 'revenue'; formulas may only reference lines |
| lines that depend on one another | circular reference: ebit -> ebitda -> ebit |
The typo case is the one worth sitting with. cgos instead of cogs is probably the most ordinary mistake anyone makes in a financial model, and under eval() it comes back as a NameError raised deep inside the evaluation of whichever period happened to run first, with a traceback through the evaluator instead of a sentence naming the line. Here the constructor refuses it, before the model has been handed any data at all. A loader that reads specs from JSON, YAML or a database inherits this for free: a loaded spec is validated exactly like a hand-written one, because validation lives in the dataclass and not in the loader.
The order is known before the numbers#
Because every computed line already knows what it reads, the statement can sort them. evaluation_order is a topological sort of the computed lines over the graph of their references, so a line always comes after everything it reads, whatever order the spec declares them in; dependency_graph exposes the same edges keyed by line id. A cycle is not something you discover during evaluation. It is CircularReferenceError, raised at construction, carrying the path that closes the loop. The search keeps its own stack rather than recursing, so a long chain of dependent lines is a long list and not a RecursionError.
This is the part with no eval() equivalent at all. With eval(), dependency order is whatever your evaluation recursion discovers at runtime, so a cycle surfaces as a stack overflow on the first period that has numbers in it, and the shape of the model stays unanswerable until it has been run. Having the graph in hand before the data arrives is what makes the rest of the library possible: months computed in order with closing balances carried into the next period's opening lines, input lines rolled up into quarters and a year by sum or by edge, and a checklist that states what the numbers must agree on, assets against what funds them, the balance sheet's cash against what the cash flow closes at, each with an explicit tolerance.
Division by zero is an answer, not an exception#
One more thing eval() decides for you. A margin line divides by revenue, and in a pre-launch month revenue is zero. eval() raises, which in practice gives you a model that cannot be computed at all because one ratio has no value. We make the absence of a value a value:
@dataclass(frozen=True)
class NotApplicable:
"""An amount a line cannot have, carrying the reason it has none."""
reason: str = "division by zero"
def __str__(self) -> str:
return f"n/a ({self.reason})"
def __bool__(self) -> bool:
return FalseIt is falsy, it carries the reason it has none, and it propagates: UnaryOp and BinaryOp both return it unchanged the moment an operand is one, and Rounding.apply leaves it untouched instead of trying to quantize it. A ratio that cannot exist this month renders as n/a (division by zero) and the rest of the statement computes.
What it costs to run#
The parser and the spec validator are two small modules, and they are the entire cost. There is no sandbox to maintain, no namespace blacklist, no list of builtins to keep stripping as Python adds them. The real cost is the ceiling. The grammar is four operations, parentheses, unary signs and references, and that is all it will be unless we change the parser. A model that wants max(0, x) or a conditional does not get it from a config change. It gets a new production in _Parser, new node types, and tests. We hold that line deliberately, because every function we add to the grammar is a new way for a formula to mean something a reviewer did not expect.
What pays for it is the testing shape. The parser is covered by property tests, whole statements by golden files, and every rejection above is an assertion on an error message rather than on an exception type, which is what matters when the person reading the error is modelling and not debugging. The library is pre-alpha: statements validate, formulas parse and evaluate, periods roll into quarters and a year, checks state what must agree. Rendering is not implemented yet.
None of this is a security feature. It is the ordinary engineering consequence of treating a formula as a language with a grammar, instead of as a string you hand to the interpreter. The security property arrives as a side effect, which is the only way we have ever seen it arrive reliably.