Toneme · Financial-agent training & evaluation data

Financial Agent RL Environments

A representative sample of ready-to-run financial-agent tasks built for post-training and evaluation. Source artifacts, executable workflows and reward evaluation are supplied together; recorded runs below show the environment in action.

Request private evaluation pack Inspect sample tasks →

Tasks: 14
Categories: 6
Models: 2
Retained slots: 84

Task-level final score distribution

Score / 100

GPT-5.6 Sol14 / 14 tasks · 42 slots

Full-credit tasks: 0/14 (0.00%)Full-credit slots: 0/42 (0.00%)

GPT-6 Astra14 / 14 tasks · 42 slots

Full-credit tasks: 1/14 (7.14%)Full-credit slots: 3/42 (7.14%)

Each observation is a task mean across retained slots. Open box: 25th–75th percentiles; middle line: median. Whiskers reach the furthest observed task scores within 1.5×IQR; open circles show all outlying tasks. Numeric label: overall task-weighted mean.

01 / Inspect a real task

Look past the aggregate

Redacted model-submitted artifact

LNG valuation

Build a linked three-statement forecast, discounted cash-flow valuation and sensitivity analysis for an LNG receiving terminal.

Required deliverable
Formula-driven financial workbook + structured outputs

This model submission earned the score shown here. It is an inspectable example of the requested deliverable, not a reference answer.

Final score: 75.52

GPT-6 Astra · Slot 1

Inspect the recorded execution →

Inspect the redacted income statement

IS | LNG Receiving TerminalRMB million unless labelled otherwise
Line item	Year 3A	Year 4E	Year 5E	Year 6E	Year 7E	Year 8E	Year 9E
core	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld
storage	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld
ancillary	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld
revenue	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld
operating cost	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld
surcharges	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld
dispatch	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld
gna	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld
other operating	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld
ebitda	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld
depreciation	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld
amortization	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld
ebit	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld
cash interest	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld
deposit income	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld	Withheld

HTML reconstruction of the model-submitted income statement excerpt. Row labels and column headers are preserved; financial values and formulas are withheld. Typography and colors are adapted for this gallery.

02 / Workflow coverage

Six workflows. Distinct challenges

Fixed 0–100 scale

Lab focus

Mean final score by workflow

Equal task weights within each category

Category sizes differ. Overall scores weight tasks equally, not categories equally. Missing results are not zero scores.

Credit constrained

67 / 84retained slots triggered a score cap. A ceiling limits credit; it does not award it.

Success retained

3retained slots earned full credit. Successful cases belong beside constrained outcomes.

Environment included

14task environments pair executable workflows with structured reward evaluation.

03 / More samples

Where outputs fall short

Redacted task summaries

02 / Operating valuation

A forecast must respond to its inputs

Build an integrated operating forecast and valuation for a shopping mall.

Required deliverable
Linked operating forecast and valuation workbook

Earned credit constrained by a causality check

The selected Astra submission earned 36.46 points before the cap. The recorded check did not detect real numeric causality, triggering a 25-point ceiling and reducing the final score to 25.00.

Natural score: 36.46
Final score: 25.00
Score ceiling: 25.00

GPT-6 Astra · Slot 1 · Scoring receipt excerpt

This selected submission illustrates a specific scoring constraint. It does not establish a general capability ranking or replace the complete results below.

04 / Inside a recorded run

Execution trajectories

GPT-6 Astra · one retained slot per workflow

Follow six selected executions from recorded actions to retained outcomes. These examples illustrate process; they are not category-average or best-case trajectories.

GPT-6 Astra · Slot 1 · 12 Turns · 385.8 s · Final score 75.52

Inspect & plan · 2
Build & execute · 3
Check & recalculate · 2
Monitor progress · 4
Export & report · 1

Enable JavaScript to explore individual recorded steps. Summary statistics are listed below.

LNG valuation: 12 Turns, 385.8 s, Final score 75.52

DAR buyout: 20 Turns, 615.8 s, Final score 25.00

Cash-flow bridge A: 9 Turns, 210.6 s, Final score 25.00

Loan eligibility: 13 Turns, 291.1 s, Final score 100.00

Funding scenarios A: 46 Turns, 1222.2 s, Final score 34.49

Project appraisal A: 12 Turns, 302.9 s, Final score 64.73

Duration shows intervals between recorded events; it does not separate model inference from tool execution. One turn is one recorded agent step. Colors group actions by their reviewed meaning, not by an automated clustering model. Check progress reads an existing process status; Pause is a recorded pause. Action summaries are redacted; reported validation is not the private evaluation score.

05 / Inspect the evidence

Retained results

6 categories

01DCF & OperationsView tasksGPT-5.6 Sol20.002/2 tasks · 6/6 capped slotsGPT-6 Astra46.592/2 tasks · 3/6 capped slots

Model	Mean score	Cap triggered	Slot evidence
GPT-5.6 Sol	15.00	3/3	View slots Slot 1 · 10.00 Natural score: 21.26 Final score: 10.00 Score ceiling: 10.00 Valid retained submission Slot 2 · 25.00 Natural score: 43.79 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 3 · 10.00 Natural score: 28.15 Final score: 10.00 Score ceiling: 10.00 Valid retained submission
GPT-5.6 Sol	25.00	3/3	View slots Slot 1 · 25.00 Natural score: 31.87 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 2 · 25.00 Natural score: 32.01 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 3 · 25.00 Natural score: 34.69 Final score: 25.00 Score ceiling: 25.00 Valid retained submission
GPT-6 Astra	68.19	0/3	View slots Slot 1 · 75.52 Natural score: 75.52 Final score: 75.52 Score ceiling: Not triggered Valid retained submission Slot 2 · 72.00 Natural score: 72.00 Final score: 72.00 Score ceiling: Not triggered Valid retained submission · Authorized replacement Slot 3 · 57.05 Natural score: 57.05 Final score: 57.05 Score ceiling: Not triggered Valid retained submission
GPT-6 Astra	25.00	3/3	View slots Slot 1 · 25.00 Natural score: 36.46 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 2 · 25.00 Natural score: 33.79 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 3 · 25.00 Natural score: 35.72 Final score: 25.00 Score ceiling: 25.00 Valid retained submission

02LBOView tasksGPT-5.6 Sol13.504/4 tasks · 12/12 capped slotsGPT-6 Astra24.214/4 tasks · 12/12 capped slots

Model	Mean score	Cap triggered	Slot evidence
GPT-5.6 Sol	12.65	3/3	View slots Slot 1 · 10.00 Natural score: 21.27 Final score: 10.00 Score ceiling: 10.00 Valid retained submission Slot 2 · 10.00 Natural score: 14.78 Final score: 10.00 Score ceiling: 10.00 Valid retained submission Slot 3 · 17.95 Natural score: 17.95 Final score: 17.95 Score ceiling: 25.00 Valid retained submission
GPT-5.6 Sol	10.00	3/3	View slots Slot 1 · 10.00 Natural score: 18.38 Final score: 10.00 Score ceiling: 10.00 Valid retained submission Slot 2 · 10.00 Natural score: 32.94 Final score: 10.00 Score ceiling: 10.00 Valid retained submission Slot 3 · 10.00 Natural score: 19.68 Final score: 10.00 Score ceiling: 10.00 Valid retained submission
GPT-5.6 Sol	16.37	3/3	View slots Slot 1 · 10.00 Natural score: 26.46 Final score: 10.00 Score ceiling: 10.00 Valid retained submission Slot 2 · 19.74 Natural score: 19.74 Final score: 19.74 Score ceiling: 25.00 Valid retained submission Slot 3 · 19.36 Natural score: 19.36 Final score: 19.36 Score ceiling: 25.00 Valid retained submission
GPT-5.6 Sol	15.00	3/3	View slots Slot 1 · 25.00 Natural score: 30.88 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 2 · 10.00 Natural score: 13.82 Final score: 10.00 Score ceiling: 10.00 Valid retained submission Slot 3 · 10.00 Natural score: 10.53 Final score: 10.00 Score ceiling: 10.00 Valid retained submission
GPT-6 Astra	25.00	3/3	View slots Slot 1 · 25.00 Natural score: 33.21 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 2 · 25.00 Natural score: 27.86 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 3 · 25.00 Natural score: 25.96 Final score: 25.00 Score ceiling: 25.00 Valid retained submission
GPT-6 Astra	24.14	3/3	View slots Slot 1 · 23.58 Natural score: 23.58 Final score: 23.58 Score ceiling: 25.00 Valid retained submission Slot 2 · 24.31 Natural score: 24.31 Final score: 24.31 Score ceiling: 25.00 Valid retained submission · Authorized replacement Slot 3 · 24.53 Natural score: 24.53 Final score: 24.53 Score ceiling: 25.00 Valid retained submission
GPT-6 Astra	25.00	3/3	View slots Slot 1 · 25.00 Natural score: 31.85 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 2 · 25.00 Natural score: 42.71 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 3 · 25.00 Natural score: 42.55 Final score: 25.00 Score ceiling: 25.00 Valid retained submission
GPT-6 Astra	22.70	3/3	View slots Slot 1 · 25.00 Natural score: 25.37 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 2 · 19.44 Natural score: 19.44 Final score: 19.44 Score ceiling: 25.00 Valid retained submission · Authorized replacement Slot 3 · 23.66 Natural score: 23.66 Final score: 23.66 Score ceiling: 25.00 Valid retained submission

03Cash FlowView tasksGPT-5.6 Sol28.502/2 tasks · 4/6 capped slotsGPT-6 Astra27.222/2 tasks · 5/6 capped slots

Model	Mean score	Cap triggered	Slot evidence
GPT-5.6 Sol	29.08	2/3	View slots Slot 1 · 25.00 Natural score: 37.14 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 2 · 25.00 Natural score: 37.84 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 3 · 37.23 Natural score: 37.23 Final score: 37.23 Score ceiling: Not triggered Valid retained submission
GPT-5.6 Sol	27.91	2/3	View slots Slot 1 · 21.69 Natural score: 21.69 Final score: 21.69 Score ceiling: 25.00 Valid retained submission Slot 2 · 25.00 Natural score: 38.22 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 3 · 37.05 Natural score: 37.05 Final score: 37.05 Score ceiling: Not triggered Valid retained submission
GPT-6 Astra	29.43	2/3	View slots Slot 1 · 25.00 Natural score: 37.86 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 2 · 38.30 Natural score: 38.30 Final score: 38.30 Score ceiling: Not triggered Valid retained submission Slot 3 · 25.00 Natural score: 35.76 Final score: 25.00 Score ceiling: 25.00 Valid retained submission
GPT-6 Astra	25.00	3/3	View slots Slot 1 · 25.00 Natural score: 34.59 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 2 · 25.00 Natural score: 34.35 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 3 · 25.00 Natural score: 34.59 Final score: 25.00 Score ceiling: 25.00 Valid retained submission

04Loan TapeView tasksGPT-5.6 Sol10.002/2 tasks · 6/6 capped slotsGPT-6 Astra62.502/2 tasks · 3/6 capped slots

Model	Mean score	Cap triggered	Slot evidence
GPT-5.6 Sol	10.00	3/3	View slots Slot 1 · 10.00 Natural score: 31.90 Final score: 10.00 Score ceiling: 10.00 Valid retained submission Slot 2 · 10.00 Natural score: 34.43 Final score: 10.00 Score ceiling: 10.00 Valid retained submission Slot 3 · 10.00 Natural score: 34.43 Final score: 10.00 Score ceiling: 10.00 Valid retained submission
GPT-5.6 Sol	10.00	3/3	View slots Slot 1 · 10.00 Natural score: 17.14 Final score: 10.00 Score ceiling: 10.00 Valid retained submission Slot 2 · 10.00 Natural score: 25.00 Final score: 10.00 Score ceiling: 10.00 Valid retained submission Slot 3 · 10.00 Natural score: 24.29 Final score: 10.00 Score ceiling: 10.00 Valid retained submission
GPT-6 Astra	100.00	0/3	View slots Slot 1 · 100.00 Natural score: 100.00 Final score: 100.00 Score ceiling: Not triggered Valid retained submission Slot 2 · 100.00 Natural score: 100.00 Final score: 100.00 Score ceiling: Not triggered Valid retained submission Slot 3 · 100.00 Natural score: 100.00 Final score: 100.00 Score ceiling: Not triggered Valid retained submission
GPT-6 Astra	25.00	3/3	View slots Slot 1 · 25.00 Natural score: 25.00 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 2 · 25.00 Natural score: 25.00 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 3 · 25.00 Natural score: 25.00 Final score: 25.00 Score ceiling: 25.00 Valid retained submission

05M&A & FundingView tasksGPT-5.6 Sol9.342/2 tasks · 1/6 capped slotsGPT-6 Astra41.002/2 tasks · 6/6 capped slots

Model	Mean score	Cap triggered	Slot evidence
GPT-5.6 Sol	12.44	1/3	View slots Slot 1 · 5.42 Natural score: 5.42 Final score: 5.42 Score ceiling: Not triggered Valid retained submission Slot 2 · 6.91 Natural score: 6.91 Final score: 6.91 Score ceiling: Not triggered Valid retained submission Slot 3 · 25.00 Natural score: 33.12 Final score: 25.00 Score ceiling: 25.00 Valid retained submission
GPT-5.6 Sol	6.23	0/3	View slots Slot 1 · 6.47 Natural score: 6.47 Final score: 6.47 Score ceiling: Not triggered Valid retained submission Slot 2 · 7.95 Natural score: 7.95 Final score: 7.95 Score ceiling: Not triggered Valid retained submission Slot 3 · 4.28 Natural score: 4.28 Final score: 4.28 Score ceiling: Not triggered Valid retained submission
GPT-6 Astra	38.25	3/3	View slots Slot 1 · 34.49 Natural score: 34.49 Final score: 34.49 Score ceiling: 49.00 Valid retained submission Slot 2 · 38.64 Natural score: 38.64 Final score: 38.64 Score ceiling: 49.00 Valid retained submission Slot 3 · 41.63 Natural score: 41.63 Final score: 41.63 Score ceiling: 49.00 Valid retained submission
GPT-6 Astra	43.74	3/3	View slots Slot 1 · 42.25 Natural score: 42.25 Final score: 42.25 Score ceiling: 49.00 Valid retained submission Slot 2 · 44.49 Natural score: 44.49 Final score: 44.49 Score ceiling: 49.00 Valid retained submission Slot 3 · 44.49 Natural score: 44.49 Final score: 44.49 Score ceiling: 49.00 Valid retained submission

06IRR & ComparablesView tasksGPT-5.6 Sol19.682/2 tasks · 6/6 capped slotsGPT-6 Astra38.852/2 tasks · 3/6 capped slots

Model	Mean score	Cap triggered	Slot evidence
GPT-5.6 Sol	21.31	3/3	View slots Slot 1 · 24.32 Natural score: 24.32 Final score: 24.32 Score ceiling: 25.00 Valid retained submission Slot 2 · 14.59 Natural score: 14.59 Final score: 14.59 Score ceiling: 25.00 Valid retained submission Slot 3 · 25.00 Natural score: 25.54 Final score: 25.00 Score ceiling: 25.00 Valid retained submission
GPT-5.6 Sol	18.06	3/3	View slots Slot 1 · 20.68 Natural score: 20.68 Final score: 20.68 Score ceiling: 25.00 Valid retained submission Slot 2 · 25.00 Natural score: 31.62 Final score: 25.00 Score ceiling: 25.00 Valid retained submission Slot 3 · 8.51 Natural score: 8.51 Final score: 8.51 Score ceiling: 25.00 Valid retained submission
GPT-6 Astra	65.14	0/3	View slots Slot 1 · 64.73 Natural score: 64.73 Final score: 64.73 Score ceiling: Not triggered Valid retained submission Slot 2 · 64.73 Natural score: 64.73 Final score: 64.73 Score ceiling: Not triggered Valid retained submission Slot 3 · 65.95 Natural score: 65.95 Final score: 65.95 Score ceiling: Not triggered Valid retained submission
GPT-6 Astra	12.57	3/3	View slots Slot 1 · 10.95 Natural score: 10.95 Final score: 10.95 Score ceiling: 25.00 Valid retained submission Slot 2 · 14.59 Natural score: 14.59 Final score: 14.59 Score ceiling: 25.00 Valid retained submission Slot 3 · 12.16 Natural score: 12.16 Final score: 12.16 Score ceiling: 25.00 Valid retained submission · Authorized replacement

Search locates tasks without recalculating category means. Sorting describes this selected sample; it is not a fair capability ranking.

06 / Credit and constraints

Before and after the cap

Model

View all category values

Workflow	Model	Natural score	Final score
DCF & Operations	GPT-5.6 Sol	31.96	20.00
DCF & Operations	GPT-6 Astra	51.76	46.59
LBO	GPT-5.6 Sol	20.48	13.50
LBO	GPT-6 Astra	28.75	24.21
Cash Flow	GPT-5.6 Sol	34.86	28.50
Cash Flow	GPT-6 Astra	35.91	27.22
Loan Tape	GPT-5.6 Sol	27.86	10.00
Loan Tape	GPT-6 Astra	62.50	62.50
M&A & Funding	GPT-5.6 Sol	10.69	9.34
M&A & Funding	GPT-6 Astra	41.00	41.00
IRR & Comparables	GPT-5.6 Sol	20.88	19.68
IRR & Comparables	GPT-6 Astra	38.85	38.85

Natural score is the recorded pre-cap score, not accuracy. Other scoring gates may already affect it. A hard cap limits the final score; it never adds credit. Hover or tap a bar for task and slot scores.

07 / Methodology

Evidence before credit

Our ready-to-run RL environment connects financial tasks to measurable reward. Natural scores capture earned quality; hard-cap gates constrain credit when critical requirements are unmet. Recorded execution and frozen submissions keep the result inspectable.

Evidence before credit

Toneme · Evaluation methodology

01Task & artifacts
02Recorded execution
03Frozen submission
04Evaluation
Natural scoreQuality earned
AND
Hard-cap gatesCredit constrained
Final score
A ceiling, never a bonus.
05Auditable result

Execution traceSteps, tools, recorded time

Controlled replayDocker configuration · isolated runs

Evidence bindingSubmission checksums · scoring receipts

Checks vary by task family. Infrastructure failures remain unscored.

Evaluation design overview. Checks and replay mechanisms vary by task family.

A complete RL environment. Ready to run

We provide the task environment, source artifacts, Docker configuration, execution contract and reward evaluation together. With the required runtime and images installed, researchers can run agents, collect trajectories and score their submissions through the supplied workflow. The evaluation pack includes the environment and grading components; this gallery shows public evidence.

Reward quality. Constrain shortcuts

Natural score records earned task credit; hard-cap gates constrain credit when critical requirements are unmet. Applicable checks examine formulas, recalculation and input responsiveness, reducing reliance on plausible static answers. These mechanisms address specific failure modes; they do not establish complete resistance to reward hacking.

Control the run. Preserve the submission

Docker configurations and recorded runtime profiles support inspection and replay. The Type06 replay uses a fresh container, read-only task inputs and a separate output directory, with submission checksums checked before and after execution. Recalculation and formula-cache observations help assess stale workbook values in that family. These controls reduce residual-state risks; replay still depends on the required images and calculation environment.

Keep the evidence attached

Selected Astra runs include recorded steps and timestamps. Submission checksums, scoring receipts and the acceptance ledger bind retained results to their evidence. Under the recorded recovery profiles, infrastructure failures remain unscored; they are not converted into model failures or used to reroll valid low scores.

One calculation, throughout the page

Scores use a 0–100 scale. We average retained slots within a task, then weight covered tasks equally. Category means apply the same rule within each category. Coverage is shown explicitly. Different task counts and selection protocols must be considered before comparing models.

Evaluate training efficacy in a scoped pilot

This gallery is an illustrative sample, not a statistical claim about training lift or model ranking. Training efficacy should be evaluated on a scoped pilot against the target model and objective.

Continue the technical conversation

Discuss a private evaluation pack

Interested in the private evaluation pack or a broader task set? Contact us by email to discuss your model, workflow and evaluation objective.

Request private evaluation pack

jaredleung@toneme.ai

Financial Agent RL Environments

Task-level final score distribution

Look past the aggregate

LNG valuation

Six workflows. Distinct challenges

Mean final score by workflow

Where outputs fall short

A forecast must respond to its inputs

Earned credit constrained by a causality check

A ceiling does not award the points beneath it

Low scores remain low after a cap is applied

Execution trajectories

Retained results

Before and after the cap

Evidence before credit

Evidence before credit

A complete RL environment. Ready to run

Reward quality. Constrain shortcuts

Control the run. Preserve the submission

Keep the evidence attached

One calculation, throughout the page

Evaluate training efficacy in a scoped pilot

Discuss a private evaluation pack