If You Can’t Price It, Don’t Build It: How to Measure AI ROI
Authored by Kwesi Apponsah, PhD, Head of Astra Labs
Sometime this year, a CFO will be asked to renew an AI contract. The demo that won the work was impressive, and there was strong momentum internally to get the project moving. The pilot launched a year ago, but the forward deployed engineering team is still on-site.
Somewhere along the way, the scope drifted and now it’s unclear what value this AI initiative is driving within the organization.
Unfortunately, that CFO is in good company. In an IBM survey of 2,000 CEOs, respondents reported that just 25% of AI initiatives delivered the ROI they expected. And when it comes to getting AI work into production, MIT’s NANDA found that a paltry 5% of custom enterprise AI tools ever get there.
Although there is a critical need for AI to solve key operational and data-driven problems, enterprises are walking away from their AI initiatives. For example, S&P Global Market Intelligence found that 42% of companies abandoned their AI initiatives last year (up from 17% prior year).
Surprisingly, spend on AI continues to rise. The same CEOs in IBM’s study, for example, expect the growth rate of their AI investment to more than double over the next two years. More capital is flowing into a technology that, based on current evidence, turns a small number of projects into real returns.
Speaking from firsthand experience, most of the AI initiatives I’ve observed that never took off are a result of no one defining “done”. Instead, the work gets mapped into a long list of pain points: fewer manual steps, less backlog, faster turnaround. The problem is that no one on the team described what the work would look like once agents were doing part of it, or the right KPI for a project or workflow to be deemed successful. To be clear, the target doesn’t need to be ambitious; “we expect coverage will move from 5% to 25%.” If that shift pays back the investment 2x, it’s a starting point worth funding.
What matters is that someone outlines the target before the build begins. This is how Teragonia works. Every engagement starts with the KPIs the business runs on, and the levers that drive value creation. The data, the workflows and the technology come after, and they’re built to move those numbers.
AI projects require the same order.
With Astra Labs, we bring that discipline to every AI build: name the target first, then decide what to build to get there.
Start with the target, then decide how you’ll get there
Most stalled projects start at the beginning, when instead they should start at the end, or the expected outcome. Leadership sees the bells and whistles of a new AI tool and goes looking for a place or a process to embed it. Gartner describes most agentic AI projects today as early experiments driven by hype and often misapplied, estimating that only ~130 of the thousands of vendors selling agentic AI offer something real.
Not to mention, the wrong project gets funded. MIT found that about half of generative AI budgets go to sales and marketing, where results can be demonstrated quickly. However, the largest returns consistently come from back-office automation. The challenge is that most enterprises review AI work in context of hours saved, which is a weak unit of measurement. Freed-up time only becomes dollars saved when the company acts on it, such as absorbing that growth without hiring additional headcount, moving people to more strategic work, or cutting overtime and contractor spend.
When it comes to AI initiatives, a stricter standard is needed. Instead, starting from the target or the outcome that you are expected to drive helps mitigate this problem.
For example, a “result” matters when a financial number changes, and the human-in-the-loop confirms this. Metrics such as: EBITDA, days of sales outstanding, cost per claim, write-offs, cost of a late close, all qualify as strong metrics of ROI measurement.
One way to think about what to measure is based on the actions required from the work. For example: how often does this type of work happen? How many hands does it pass through before it’s complete? What does a delay or an error in the work cost?
High-volume work that passes through several teams is the richest ground. In finance, applying customer payments to open invoices, reconciling bank accounts and processing payables all fit that profile, and feed the short-term cash forecast that a CFO reviews every week.
How often the output gets used matters, too. A finance team reads a supplier contract closely once, at signing. A healthcare provider is paid against its payer contracts on every claim it submits, so checking contract terms there can recover underpayments inside the claim window. The extraction automation work in both examples is essentially the same, but it’s worth more in the healthcare example.
Price the result before you build
Once the KPI targets are selected, they require a dollar value before any build begins. Multiply the cost per unit by annual volume, then by the share you can realistically remove. Count only savings that show up in a financial number, not hours saved. Compare that number with the cost of building the solution and running it for a year. If the return isn’t clearly bigger, don’t build it. Most AI projects skip this step, and a project that starts without a number rarely finds one later.
After selecting the targets and putting a cost against them, you’ll also need to:
- Appoint a senior leader who will own all decisions and be responsible for driving the work to completion
- Record the baseline before a build starts, with the same definitions and KPIs that will be used to assess success of the initiative
These steps sound obvious, and most leadership teams believe they account for them. However the numbers tell a different story: in IBM’s study, 68% of CEOs said they had clear metrics for innovation ROI, yet three-quarters of initiatives still missed their expected return.
Having metrics isn’t the same as pricing the target before anything gets built.
Split the work between code, models and people
Watch a credit analyst work a loan file, and you’ll notice how little of the job is actual judgment. Most of the hours go to collecting the borrower’s documents and making sure the numbers in them agree. The analyst downloads tax returns and financial statements from one system, types the figures into the bank’s spreadsheet, and confirms that the revenue on the tax return matches the revenue in the audited accounts. The real decision is whether this borrower can carry this debt and is often where they spend the least amount of time.
Mapping that work helps identify where each tool belongs. Gathering and checking follow fixed rules, and ordinary software is the right tool for fixed rules. It gives the same answer every time, and you can test it line-by-line. Using a language model for that work adds cost and higher potential for error. Models earn their cost on messy inputs like a scanned tax return, a footnote in a lease schedule or a covenant on page 40 of a loan agreement. A model reads those faster than any analyst, and it can report how confident it is in each number it pulls out. Few companies make enough use of this.
Set a confidence threshold, send anything below it to the analyst, and their time goes to the few numbers that genuinely need a second look.
A person still has to approve the final decision. Sooner or later, a compliance team, an auditor or a regulator will want to know who signed off on a loan, payment or claim, and “the system” isn’t an acceptable answer. Projects that don’t make this clear often end up stuck in risk review for months. Deciding up front where people sign off makes that review go faster, because the risk and compliance teams can see exactly where a human is still making the call.
In the lending example, the success criteria are agreed before any work starts. The numbers the system extracts from borrower financial statements should match what analysts entered manually on a sample of past loan files, within an agreed tolerance.
This also shows where most of the build cost sits. Connecting to systems and extracting data from documents is work that largely carries over from one project to the next, so it gets cheaper over time. The workflow layer on top is different. Routing tasks, collecting approvals, managing worklists and writing results back to the system of record usually must be built again for each client, and that’s where buyers should push a delivery partner on what they’ve already built.
It’s also worth asking whether a partner’s second build of a workflow actually cost less than its first.
Maintain the right to walk away
When an AI project disappoints, people usually blame the technology. In many cases, the problem goes back to how the the contract as written. Most AI contracts pay the vendor for time spent rather than results, and they never define what a successful outcome looks like.
None of this means that companies should build everything themselves. For example, MIT found that AI tools bought from specialist vendors or built with partners succeed about twice as often as those built in-house. Vendors know this, which is why they are sending their own engineers into client businesses to do the work. Lightcast data reported by Fortune shows that job postings for these forward-deployed engineers rose more than 1,000% in the first eight months of 2026 (vs 2025). That approach gets a working system live quickly, but the people who understand it work for the vendor. Over time, the company can lose the ability to change, fix or shut down what it paid for without them.
A buyer can protect against that, but it’s essential to do so at the start of an engagement. The contract should spell out what will be built, what is excluded, and which data the company is responsible for providing. It should define success as a measurable test on real cases, with a fixed period to run it. It should outline who pays when problems come from the company’s own data. It should also guarantee that the system runs on the company’s own infrastructure, that the company’s staff can modify it, and that the code, data model and documentation belong to the company when the engagement ends.
This is how we set up every Astra Labs engagement. We build inside the client’s own infrastructure and data, alongside their teams, so their people know how the system works by the time it goes live. Every engagement is priced as a fixed fee and held to a guaranteed outcome, and every build is owned by the client from day one.
Last year, the average company scrapped 46% of its AI proofs of concept (S&P Global). Many of those didn’t need to be thrown out. They needed someone to notice they were off track and change course. A clear target tells you early when a project is heading the wrong way, and owning what was built means you can adjust and iterate instead of starting over.
What to ask before approving your next AI build
Most of what separates the AI projects that pay back from the ones that drift can be settled in a single meeting, before any money is committed. None of these questions are technical, and the CFO is usually the right person to ask them.
- Which financial number will this project change, and who is accountable for it? If the answer is a broad benefit such as efficiency or better insight, the project has no target, and a year from now nobody will be able to say whether it worked.
- Where does that number stand today? Record it before the build, measured exactly the way it will be measured afterward. Without that starting point, any improvement is a matter of opinion.
- What is changing it worth, and what will the project cost? Compare the dollar value over a year with the cost to build and run it over the same period. If the gap is thin, the project should go back for rethinking before anyone signs a contract.
- Which steps need AI, and which don’t? Ordinary software should handle the steps that follow fixed rules, AI should handle the messy inputs, and a person should review and approve the result. A proposal that sends every step through an AI model is usually more expensive and less reliable than it needs to be.
- How will we know it works? Agree on a test before the build starts, and run it on real cases from the business.
- What result would make us stop? Write it down while everyone is still objective.
- Could our own people run it if the vendor’s engineers left tomorrow? If the honest answer is no, the cost of that dependency belongs in the business case.
A project that can answer the above is ready for funding.
Interested in scheduling an assessment to see if your project is the right fit?
Reach out: info@teragonia.com
About Astra Labs
Astra Labs is a business unit of Teragonia that brings applied AI to complex enterprise problems and workflows. Astra Labs deploys directly inside every client’s environment and moves fast from the first conversation to AI running in production. Every engagement is priced as a fixed fee and held to a guaranteed outcome, and every Astra Labs build is owned by the client.
An AI engineering leader and systems architect with a PhD in Aerospace Engineering from the University of Toronto and over a decade of experience across the full delivery lifecycle, Kwesi turns frontier model capability into governed systems that hold up under real operational load. Throughout his career, he has led enterprise technical teams and delivered production-grade, governed AI platforms across major financial institutions, pensions, and utilities. At Teragonia, he combines computational depth with enterprise delivery experience to help private equity firms and their portfolio companies capture value through applied AI across the full investment lifecycle.
Sources:
McKinsey & Company, “The state of AI in 2026: On the road to ROI”, August 25, 2026. Online survey of 1,719 respondents in 97 countries, May 4 to June 8, 2026.
IBM Institute for Business Value, “IBM Study: CEOs Double Down on AI While Navigating Enterprise Hurdles”, press release, May 6, 2025. Survey of 2,000 CEOs globally.
MIT NANDA, The GenAI Divide: State of AI in Business 2025, July 2025. Review of 300+ public AI deployments, 52 interviews and a survey of 153 leaders, as reported by Virtualization Review, August 19, 2025.
S&P Global Market Intelligence 2025 survey of more than 1,000 respondents in North America and Europe, as reported by CIO Dive, 2025.
Gartner, “Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027”, press release, June 25, 2025.
Preston Fore, Fortune, September 3, 2026, citing Lightcast job postings data.
Aloan, “Best Credit Memo Software for Commercial Lenders (2026)”, vendor-published, 2026.
FintechSpecs, “Best AI Credit Memo and Financial Spreading Software for Commercial Lenders”, 2026. Figures drawn from vendor-published case studies.
U.S. Bureau of Labor Statistics, Occupational Employment and Wage Statistics, Credit Analysts (SOC 13-2041), May 2025. National median annual wage $83,510.
Aloan, “How to Reduce Turnaround Time in Commercial Loan Underwriting”, vendor-published, 2026.
Sheryl Estrada, “MIT report: 95% of generative AI pilots at companies are failing”, Fortune, August 18, 2025. Reporting on MIT NANDA, The GenAI Divide: State of AI in Business 2025.
Foundational Tech Infrastructure
Our core analytics and AI platform drives informed decision-making with enhanced clarity and focus, and rapidly unlocks enterprise value
Core Features:
Connect All Your Data Sources
Integrate data from multiple source systems effortlessly
Critical for breaking down silos and creating a unified view
One Trusted Data Source
A secure, centralized cloud hub for all your data insights
Foundational for reliable decision-making and enterprise-wide alignment
Interactive Dashboards
Visualize complex data through easy-to-understand dashboards
Empowers leaders with actionable insights
Enriched Data Flow, Fully Automated
Reverse ETL capabilities enriches your data and ensures your data flows exactly where it is needed for function teams to act on
Enables real-time, action-oriented data flow
Additional AI capabilities are actively in development
Scott Briggs
BS International Business | American University of Paris
BS Computer Science | American University of Paris
Seasoned DevOps and infrastructure engineer with expertise in AWS, Kubernetes, and Terraform; led cloud migrations and scalable infrastructure projects at Sfara, FanDuel, and Kickstarter.
With over 15 years of experience in small and medium-sized startups, Scott is a seasoned expert in designing, optimizing, and maintaining robust, scalable, and secure infrastructure. He specializes in automation and embedding security from the ground up, consistently delivering reliable systems tailored to meet dynamic business requirements.
Prior to joining Teragonia, Scott made a significant impact at Sfara, where he built the company’s entire infrastructure from scratch. He engineered systems capable of supporting hundreds of thousands of users with seamless scalability, implemented automated development pipelines, and introduced observability tools to monitor and manage resources effectively. Additionally, Scott led the infrastructure team in achieving ISO27001 security certification, ensuring security was integrated into every aspect of the system and transforming it into a critical asset for business-to-business operations.
Beyond his technical expertise, Scott has a proven track record of managing and mentoring high-performing teams. As a Senior DevOps Engineer at FanDuel, he gained invaluable experience in scaling infrastructure and optimizing resources to support millions of daily users, aligning technological capabilities with organizational goals.
Jack Amedio
Master’s in Human Resources | University of Illinois
Bachelor’s in Management | Loyola University
Former Financial and Operations Manager at Houlihan Lokey, Golin Harris, and MSL Group.
Jack is a highly driven, cross functional professional with extensive experience in operations and administration.
Prior to joining Teragonia, Jack held financial and facilities management roles for Houlihan Lokey, MSL Group/Publicis, and Golin Harris in which managed and created processes and trainings for multiple functional areas ensuring operational and administrative procedures were well planned, efficient, cost-effective, and aligned with business objectives while ensuring initiatives, internal events as well as client events propelled employee and client engagement.
Jack holds undergraduate degrees from University of Illinois and Loyola University Chicago and has completed graduate certificates in Business Administration, Strategic Human Resources, and Operations at Cornell, CUNY-Buffalo, and University of Illinois and is in the process of completing a Master’s in Human Resources at Loyola University Chicago’s Quinlan School of Business.
Mason Taylor
MS Analytics | Georgia Institute of Technology
BS Management Information Systems | Oklahoma State University
Former analytics engineer at Cyderes and ConocoPhillips with a Master’s in Analytics from Georgia Institute of Technology and a Bachelor’s in Management Information Systems from Oklahoma State University
Mason is an Analytics Engineer with deep experience in data analytics, business intelligence, machine learning, and cybersecurity. He brings a proven track record of leading analytics engagements spanning architecture, insights, visualizations, and delivery.
Before joining Teragonia, Mason was a Senior Analytics Engineer at Cybersecurity MSSP CYDERES where he built a scalable, standardized, and secure analytics architecture for over 300 clients across many industries and consulted with them to deliver insights through bespoke data driven solutions. In addition, he managed the data delivery of the insight platform leveraged by the Security Operations Center to respond to incidents in a timely and effective manner.
Prior to joining CYDERES, Mason worked in ConocoPhillips’ Analytics and Innovation Center of Excellence holding varied roles within the Data Analytics organization from Data Engineering, to Business Intelligence, and Data Science. He delivered robust data solutions in all operating units for various functions including Engineering and Production, Finance, IT, and more. Including projects to standardize cost and production data across operating units.
Mason started his career at The Williams Companies in cybersecurity and transitioned to cybersecurity at ConocoPhillips where he found his passion for Data Analytics through SIEM management, detection engineering, and threat intelligence.
Grace Sun
Bachelor’s in Finance & Accounting | Georgetown University
Former analytics engineer at Houlihan Lokey and financial analytics at JP Morgan Chase with a Bachelor’s in Finance & Accounting at Georgetown University
Grace is a seasoned analytics engineer with specialized expertise in crafting and implementing analytics solutions that drive agile, informed executive decisions in M&A and value creation for private equity-backed companies.
Before joining Teragonia, Grace was a part of the data science and business analytics team at Houlihan Lokey. She has excelled in harmonizing, enriching, and analyzing data from diverse sources, providing key insights that enabled private equity investors and portfolio company executives to make rapid, data-driven decisions across the investment lifecycle. She has developed novel analytics solutions, including deal sourcing and evaluation tools for platform investments that employ a buy-and-build or de novo growth strategy, as well as post-close value creation and KPI reporting tools for operators and management teams.
Grace has also worked at JPMorgan Chase & Co. in the Global Finance and Business Management rotational program, where she built analytics solutions to evaluate banker attrition and KPI reporting within the Global Private Bank.