---
title: The runtime AI tax | qventis blog
description: Runtime-AI test tools call a model on every step of every run. How that cost compounds, what it hides, and where AI pays for itself.
---

[Skip to content](https://qventis.ai/blog/runtime-ai-tax#main)

[![qventis](https://qventis.ai/hubfs/raw_assets/public/qventis-one/images/qventis-logo.svg)![qventis](https://qventis.ai/hubfs/raw_assets/public/qventis-one/images/qventis-logo-reverse.svg)](https://qventis.ai/)

- Platform
  
  [**Qventis One**The autonomous quality platform. Learn your app once, assure every kind of quality, decide in one view.](https://qventis.ai/one)
  
  [**Qventis Engine**Plain English in, zero AI tokens at run time](https://qventis.ai/engine) [**App Model**Learn your app once, reuse it everywhere](https://qventis.ai/one/app-model) [**Quality View**One pane of glass for every release](https://qventis.ai/one/quality-view) [**Studio**Anyone can automate in plain English](https://qventis.ai/platform/studio)
  
  [**Trust & security**AI on your terms, people approve](https://qventis.ai/platform/trust) [**Agent gateway**Coding agents run tests under policy](https://qventis.ai/platform/agent-gateway) [**Integrations**Fits your CI, ALM and chat tools](https://qventis.ai/platform/integrations) [**Editions**Start small, grow into One](https://qventis.ai/editions)
  
  How it works[1Learn your app](https://qventis.ai/one/app-model)[2Write plain English](https://qventis.ai/platform/studio)[3Run every engine](https://qventis.ai/engine)[4Decide in one view](https://qventis.ai/one/quality-view)
- Engines
  
  [**Every kind of quality, one flow**Eight engines share one model of your app and report into one view.](https://qventis.ai/one)
  
  [**Functional**Web, API, desktop and mobile](https://qventis.ai/engines/functional) [**Performance**Speed budgets on real journeys](https://qventis.ai/engines/performance) [**Security**Roles, sessions and exposure](https://qventis.ai/engines/security) [**Service virtualization**Test before every API is ready](https://qventis.ai/engines/virtualization)
  
  [**Test data**Safe, ready data for every run](https://qventis.ai/engines/test-data) [**Data & ETL**Numbers that match end to end](https://qventis.ai/engines/data-etl) [**Visual**Catch what users would see](https://qventis.ai/engines/visual) [**Accessibility**WCAG checks with an audit trail](https://qventis.ai/engines/accessibility)
  
  Packaged-app packs[Oracle](https://qventis.ai/engines/packaged-apps)[SAP](https://qventis.ai/engines/packaged-apps)[Salesforce](https://qventis.ai/engines/packaged-apps)[Workday](https://qventis.ai/engines/packaged-apps)[See all packs](https://qventis.ai/engines/packaged-apps)
- Solutions
  
  [**Release in confidence**Certify every vendor upgrade and configuration change before your business feels it.](https://qventis.ai/platforms)
  
  Enterprise platforms
  
  [Salesforce](https://qventis.ai/platforms/salesforce)[SAP](https://qventis.ai/platforms/sap)[Oracle](https://qventis.ai/platforms/oracle)[Workday](https://qventis.ai/platforms/workday)[ServiceNow](https://qventis.ai/platforms/servicenow)[Dynamics 365](https://qventis.ai/platforms/dynamics-365)[nCino](https://qventis.ai/platforms/ncino)[Coupa](https://qventis.ai/platforms/coupa)
  
  [**Industries**Banking, insurance, healthcare and more](https://qventis.ai/industries)
  
  By need
  
  [**Release testing**Run only what a change affects](https://qventis.ai/solutions/release-testing) [**Legacy desktop**Apps automation never reached](https://qventis.ai/solutions/legacy-desktop) [**Quality CoE**One standard across the business](https://qventis.ai/solutions/quality-coe)
  
  Certify an upgrade[1Release notes in](https://qventis.ai/platforms)[2Impact on your app](https://qventis.ai/one/app-model)[3Run in sandbox](https://qventis.ai/engine)[4Sign off](https://qventis.ai/one/quality-view)
- [Customers](https://qventis.ai/customers)
- Company
  
  [**About qventis**Why we built one platform](https://qventis.ai/about) [**Blog**Field notes on quality, AI and agents](https://qventis.ai/blog) [**Trust center**How we protect your data](https://qventis.ai/platform/trust) [**Contact**Talk to a person, not a bot](https://qventis.ai/contact)
  
  Prefer email?[contact@qventis.ai](mailto:contact@qventis.ai)

[Book a demo](https://qventis.ai/contact#demo)

[Blog](https://qventis.ai/blog)/

# The runtime AI tax

Runtime-AI test tools call a model on every step of every run. How that cost compounds, what it hides, and where AI pays for itself.

qventis teamOctober 11, 2026

A new class of test tools asks a language model to read each step while the test runs. The model looks at the page, decides what "click checkout" means right now, and acts. No locators to maintain. No scripts to fix. In a demo it feels like the end of test maintenance.

Then the suite grows, the pipeline runs on every merge, and a new line appears in the budget. We call it the runtime AI tax: a charge paid on every step, of every test, of every run, in every environment, whether or not anything in the app has changed.

## Where the tax comes from

A runtime-AI step works roughly like this. The tool captures the current state of the app, as a screenshot, a page snapshot or both. It sends that state to a model with the instruction. The model reasons about it and returns an action. The tool performs the action and repeats for the next step.

Every one of those round trips consumes tokens. The count grows with the size of the page, the number of steps, the number of retries and the length of the instructions. Directional numbers are starting to appear. One vendor blog from September 2026 reported that an eight-step login flow used about 89,000 tokens through a browser MCP server, against about 24,000 through a command-line approach. Treat those figures as directional, since they come from one vendor and one flow. The shape is what matters: a short, boring flow can cost tens of thousands of tokens, every time it runs.

## Do the multiplication

Token prices keep falling, so it is tempting to ignore this. Volume is the problem. Take a typical enterprise suite:

- 2,000 regression tests, averaging 15 steps each
- run on three browsers for every merge to main
- 20 merges on a busy day

That is 90,000 model-interpreted steps per pipeline run and about 1.8 million on a busy day. Add retries on flaky steps, nightly runs in two more environments and a second suite for the mobile app. The per-token price can halve and the bill will still grow, because test volume grows faster than prices fall. Teams that succeed with automation run more tests, more often. Under a runtime-AI model, success is exactly what makes the bill climb.

## Cost is only the visible part

The invoice is easy to see. The other costs show up in the pipeline and in audits.

- **Latency.** Every step waits for a model round trip. A suite that once ran in minutes can stretch into a much longer gate on every merge.
- **Nondeterminism.** The same instruction on the same page can be interpreted differently on different days. A test that passed on Tuesday fails on Wednesday with no change to the app, and someone has to find out why.
- **Model drift.** When the provider updates a model, your suite's behavior can change overnight. Nobody touched the tests, yet they act differently.
- **Audit.** When a regulator or a release manager asks what a test actually did, "the model decided" is a weak answer. You need the exact action taken at each step, every run.
- **Data exposure.** Screenshots of test environments travel to a model provider on every step. If those environments hold realistic customer data, that is a new data flow to approve.

These concerns are widespread. The World Quality Report 2025-26 from Capgemini found that 60% of organizations cite hallucination and reliability as a concern with generative AI in quality engineering. Gartner predicted in June 2025 that over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls. Runtime-AI testing touches all three.

## Put the model where it pays once

AI still has a real place in testing. The placement is what matters. There are two very different moments when a model can help.

The first is authoring time. A model can draft a test from a user story, suggest the next step, or turn a recording into readable sentences. That work happens once per test. A person reviews it, and the result is stored.

The second is run time. Here the model repeats the same interpretation thousands of times, producing nothing new unless the app has changed.

A better split is to use AI while writing and maintaining tests, then compile each approved test into deterministic code that runs with no model at all. The plain-English steps below are an example. They are written once, reviewed once and compiled once.

**Checkout total is correct**Web, 4 steps

1. Open the 'Storefront' app
2. Type '' into the 'Email' field
3. Click the 'Place order' button
4. Check that 'Total' shows '42.00' within 2 seconds

When the app changes and a step fails, a model can be called to propose a fix, such as a renamed button. A person approves it, the test is recompiled, and the next thousand runs cost nothing extra. AI spend now scales with how often your app changes, which is a number you control. How often you test, a number you want to grow, stops affecting the AI bill.

> Pay for intelligence when something changes. Pay nothing when nothing has.

## Questions to ask any test tool

Whatever you use today, these questions show quickly where the AI sits and who pays for it.

1. How many model calls does one step make at run time? Zero is a valid answer.
2. If the model provider is down, do my tests still run?
3. If I replay yesterday's run, will it take exactly the same actions?
4. Which data leaves my environment during a run, and where does it go?
5. Can I switch AI off for one project and keep the suite running?
6. How is AI usage reported: per run, per step, per test?
7. When a test is healed, who approves the change, and is it recorded?

Clear answers to these questions are a good sign. Vague answers usually mean the cost and the risk are yours.

## A simple rule for budgeting

Separate two budgets. One covers the intelligence you use to create and maintain tests. It should track the rate of change in your apps. The other covers execution. It should track compute and nothing else. If your execution budget includes model tokens, you are paying the runtime AI tax, and it will rise every time your testing improves.

This is the approach behind Qventis One. Its core, the Qventis Engine, compiles plain-English steps once and runs them deterministically with zero AI tokens at run time, across web, desktop and packaged apps. AI helps write tests and propose heals, and a person approves every change. [Book a demo](https://qventis.ai/contact#demo) to see your suite run without the tax.

**Sources**

1. Capgemini, World Quality Report 2025-26. [capgemini.com/resources/world-quality-report-2025](https://www.capgemini.com/resources/world-quality-report-2025)
2. Gartner, "Gartner predicts over 40% of agentic AI projects will be canceled by end of 2027," press release, June 25, 2025. [gartner.com](https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027)
3. Vendor blog comparing browser MCP and CLI token use for an eight-step login flow, September 2026. Directional figures from a single flow.

**Qventis One** learns your app once and runs every kind of quality check in one flow, with zero AI tokens at run time.

[Book a demo](https://qventis.ai/contact#demo)

![qventis](https://qventis.ai/hubfs/raw_assets/public/qventis-one/images/qventis-logo-reverse.svg)

Qventis One is the autonomous quality platform from qventis.ai. Learn your app once. See every kind of quality in one place. Change nothing without your approval.

[contact@qventis.ai](mailto:contact@qventis.ai)

## Platform

- [Qventis One](https://qventis.ai/one)
- [Qventis Engine](https://qventis.ai/engine)
- [App Model](https://qventis.ai/one/app-model)
- [Quality View](https://qventis.ai/one/quality-view)
- [Studio](https://qventis.ai/platform/studio)
- [Agent gateway](https://qventis.ai/platform/agent-gateway)
- [Editions](https://qventis.ai/editions)

## Engines

- [Functional](https://qventis.ai/engines/functional)
- [Performance](https://qventis.ai/engines/performance)
- [Security](https://qventis.ai/engines/security)
- [Service virtualization](https://qventis.ai/engines/virtualization)
- [Test data](https://qventis.ai/engines/test-data)
- [Data & ETL](https://qventis.ai/engines/data-etl)
- [Visual](https://qventis.ai/engines/visual)
- [Accessibility](https://qventis.ai/engines/accessibility)

## Solutions

- [Enterprise platforms](https://qventis.ai/platforms)
- [Industries](https://qventis.ai/industries)
- [Legacy desktop](https://qventis.ai/solutions/legacy-desktop)
- [Release testing](https://qventis.ai/solutions/release-testing)
- [Quality CoE](https://qventis.ai/solutions/quality-coe)

## Company

- [About](https://qventis.ai/about)
- [Customers](https://qventis.ai/customers)
- [Blog](https://qventis.ai/blog)
- [Trust center](https://qventis.ai/platform/trust)
- [Contact](https://qventis.ai/contact)
- [Legal](https://qventis.ai/legal)

© 2026 qventis.ai. All rights reserved.Product names of third-party platforms are trademarks of their owners.

```json
{
  "@context" : "https://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "qventis team",
    "url" : "https://qventis.ai/blog/author/qventis-team"
  },
  "dateModified" : "2026-10-11T06:44:00.106Z",
  "datePublished" : "2026-10-11T06:45:58.000Z",
  "headline" : "The runtime AI tax | qventis blog",
  "mainEntityOfPage" : {
    "@id" : "https://qventis.ai/blog/runtime-ai-tax",
    "@type" : "WebPage"
  },
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject"
    }
  }
}
```