How to assess AI in a trading and risk workflow
Define the job
Before bringing AI into a trading or risk workflow, write down the decision it is meant to support. “Improve analysis” is too broad to test. “Help an analyst identify settlement exceptions for review” gives you something to examine.
Start with the consequence of error
Start with the consequence of a wrong answer. A draft summary, an exception flag and an instruction that changes a trading position need different controls. Identify who reviews the output, what evidence they can see and what the system is permitted to do.
Follow the data
Then follow the data. Which trade records, prices, contract terms or documents are required? Are the dates and versions clear? Can the reviewer distinguish a source fact from an inference? An answer that sounds plausible can still be based on an old amendment or the wrong entity.
Test the exceptions
Use a small, representative test set agreed with the people who do the work. Include an amended price, a missing field, conflicting documents and an exception that should be escalated. Protect confidential data and use only an environment approved for that information. Record the expected handling before looking at the tool’s output.
Make uncertainty visible
Ask the supplier to show how the output was produced and what happens when the evidence is insufficient. The test should allow “I cannot support that conclusion” to be the correct result. Also examine how permissions, review and the audit record fit the existing workflow.
Compare the whole workflow
Compare the proposed approach with the current process and any simpler feasible alternative. Include review effort, correction effort and the work created at the next handoff. A faster first draft does not establish that the full process is faster or safer.
Define the approval
Finally, define the approval. What task is authorised, with which data and controls? What remains outside scope? Who can stop or change the use when an assumption no longer holds?
Where Implementary fits
Implementary provides ETRM and CTRM advisory and focused decision modelling for commodity and energy trading. We develop PARALLEL and use it where it fits a scoped decision. This article is a guide to framing a buyer evaluation; it does not rank tools or claim that one product is suitable for every trading workflow.
If you are considering AI in a trading or risk process, start with the decision, the evidence and the consequence of an error. That gives the evaluation a useful job.

