By Kwesi Apponsah, PhD, Head of Astra Labs, and Allison Tolgyesi, PhD, AI/ML Engineer, Teragonia
Sometime this year, a steering committee will sit through the final readout of an AI pilot. The demo runs cleanly, the accuracy numbers look good, and the AI solution gets through the sample in a fraction of the time an analyst would need. Then someone asks whether to move to production and no one in the room can answer.
No one can answer because the sample was clean. The statements were legible, every borrower was a single entity and every field was filled in. A sample like that suits a proof of concept, which only has to show that the AI solution can do the task at all. The lender’s real queue also has scanned tax returns, covenants on page 40 and files that arrive with the financials missing. The pilot never saw those files, so it says nothing about how the AI solution will handle them.
Our first article in this series showed how to price an AI project before anyone starts building it. Its example is a commercial lender whose analysts spend about 10 hours on every credit memo. Cutting that time in half across 200 loans a year at $40 an hour saves about $40,000, which does not justify an expensive AI build.
The bigger number is speed. A credit decision that takes one to two weeks today could take two to three business days. A borrower who waits two weeks for an answer can take the deal to another lender.
These are still estimates. A pilot replaces them with measurements from the lender’s own loan files. A pilot that only shows analysts working faster has measured productivity, and productivity alone does not reach the bottom line. In McKinsey’s 2026 survey, 80% of respondents say AI improves their individual productivity, while only 37% report a positive EBIT impact. Many things can cause that gap. In our experience, a common one is that teams measured the gains on work that was easier than the real thing.
A good pilot ends with four numbers that the lender can measures itself, on a sample that models its real queue:
Those four turn the first article’s estimates into a net return and a Go / No-Go decision (Figure 1).
A proof of concept and a pilot answer different questions.
A proof of concept shows whether the AI solution can do the task. A pilot shows whether it pays off on the lender’s real work, so it needs files in the proportions they arrive. Pull real files from the queue your analysts work today and have someone at the lender choose them, so the sample matches the queue as the lender sees it.
Start with past loans where the outcome is on file. Run the AI solution on those files and compare its output with the analyst’s memo. Do the figures it extracts match what the analyst entered, within the tolerance you set? Does it catch anything the analyst’s memo left out, like a covenant or a footnote in a lease schedule? A credit officer reviews every catch, and the pilot records the loan balance at stake for each one that holds up.
No one waits for results, because the answers are already on file.
Next, run current applications through the AI solution as they arrive, while analysts work them as usual and sign every memo. Live loans show how fast the AI solution runs on an ordinary day, how it behaves when documents arrive late or incomplete, and how its output fits into an analyst’s routine. They also give you the number that matters most: days from a complete application to a finished credit memo.
Results differ by how hard the file is, so sort the loans into easy, typical and hard and include all three in the sample. Easy loans make any pilot look good. The hard ones, with scanned returns, related entities and long agreements, show whether the AI solution is worth keeping. Report each level separately, then weight the results by how often each level shows up in your real queue.
In Figure 2, the lender’s queue is 40% easy loans, 40% typical and 20% hard. Suppose the AI solution saves 60% of analyst time on easy loans, 40% on typical loans and 10% on hard loans. Across the whole queue that comes to 24 + 16 + 2, or 42% saved. A test on easy loans alone would have shown 60%, and a lender that planned around that number would end up 18 points short.
Run the AI solution next to the current process on the lender’s loan files and let the team work as usual. The AI solution drafts its own version of each credit memo, the analyst writes theirs, and a reviewer compares the two. The team’s routine does not change, so the pilot puts little at risk.
Say your analysts also change how they work in the week the AI solution goes live, and loans start closing faster. No one can tell whether the AI solution or the new process caused it. Time your analysts beforeyou start, using the same start and stop points you will use afterward. That is the baseline step from our first article.
In McKinsey’s 2026 survey, nearly three-quarters of high performers report fundamentally redesigning workflows around AI, up from 55% a year earlier. That redesign should come after the pilot, once you know which changes are worth making.
Agree what success looks like before you start
The numbers above say what the AI solution is worth. They hold only if the AI solution first meets a minimum standard on the work itself.
Write that standard down before the first loan runs, with a passing mark for each of the following:
For example, agreement might require the figures to match on at least nine of ten past loans. Add the net return the project has to clear.
A bar that lives only in people’s heads drifts toward whatever the results show, so write it down first.
Auditability matters because a reviewer who can’t see where a figure came from must rebuild it, and that time comes out of the savings. In a 2026 IDC survey,respondents reported spending about 13 hours a week reconstructing and validating AI outputs, and 71% said they would reject a tool that was 99% accurate if it could not explain its answers. Every figure in the memo should link to the page of the loan file it came from, so the reviewer can check it in one click. Time the reviewer as well and count that time in the hours per memo. If a reviewer needs 4 hours to check an AI-drafted memo that an analyst writes in 10, the saving is 6 hours. If the check takes 9 hours, the saving is 1.
With measured numbers, the 50% estimate becomes 42%. At 200 loans a year, 10 hours a memo and $40 an hour, that returns about $34,000 instead of $40,000. After $20,000 a year to run the system, which we assume for this example, the hours return about $14,000.
A lender that went to production on its proof of concept results would have expected 60% time saved. That is about $48,000 a year, or $28,000 after running costs. Weighted to the real queue, the net figure is about $14,000, half of what the proof of concept suggested.
As in the first article, the hours alone do not cover the build, so the case depends on how much faster the lender turns around a memo. Suppose easy loans reach a finished memo in two business days, typical loans in three and hard loans in seven. Weighted to the real queue, that averages about 3.4 days, down from one to two weeks today and a little above the two to three days in the first article’s estimate. To price that, take the number of deals the lender loses while borrowers wait and the revenue a typical loan earns, and record both before the pilot starts.
Don’t commit to a full pilot.
Split it into stages or checkpoints, giving every stage a condition that has to be true before the next one starts.
In lender pilot example, stage one runs past loans and continues only if the figures match the analysts’ within tolerance and the weighted time savings clear the bar. Stage two includes live loans and continues only if reviewer time and days to memo meet their passing marks on real work. Stage three widens to more loan types or branches, and ends with a production decision.
The lender can walk away at any gate, and stopping after stage one costs only the time spent in stage 1 instead of the entire project.
Give each gate a deadline. MIT’s NANDA research shows mid-market firms moving from pilot to full implementation in about 90 days, while large enterprises take nine months or longer. A gate with a date makes it harder for a project to drift into the longer timeline.
Back in the steering committee, the question about production now has an answer built on the lender’s own files, in the proportions the lender sees them: time saved on easy, typical, and hard loans, how many days a memo takes, how much a reviewer has to check, and what it’s worth after running costs. If that return doesn’t clear the bar set at the start, the lender stops at the stage it’s in.
Astra Labs runs its pilots in this order. Before the first loan runs, our team sits next to the client’s analysts and times the baseline on real files. We write the definitions down together. We also set the success bar and the stage gates with the client’s own people, and we agree on them before any build work starts. The pilot runs inside the client’s environment on the client’s own loan files, so the scanned tax return and the file with missing financials are in the sample. Each gate has a date and an owner. At the end, the client holds measured hours per memo, measured days to a finished memo and the reviewer’s time, all from its own cases. The client can put that evidence in front of its credit committee or its board. Every engagement has a fixed fee and a guaranteed outcome, and the client owns what we build.
About Astra Labs
Astra Labs is a business unit of Teragonia that brings applied AI to complex enterprise problems and workflows. Astra Labs deploys directly inside every client’s environment and moves fast from the first conversation to AI running in production. Every engagement is priced as a fixed fee and held to a guaranteed outcome, and every Astra Labs build is owned by the client.
An AI engineering leader and systems architect with a PhD in Aerospace Engineering from the University of Toronto and over a decade of experience across the full delivery lifecycle, Kwesi turns frontier model capability into governed systems that hold up under real operational load. Throughout his career, he has led enterprise technical teams and delivered production-grade, governed AI platforms across major financial institutions, pensions, and utilities. At Teragonia, he combines computational depth with enterprise delivery experience to help private equity firms and their portfolio companies capture value through applied AI across the full investment lifecycle.
Allison is a generative AI and LLM expert who has developed AI solutions at PwC for Canada’s leading financial institutions, specializing in intelligent automation and production-ready AI applications. She holds a PhD in Biomedical Engineering from the University of Toronto and a BASc in Biomedical Mechanical Engineering from the University of Ottawa.
Sources:
Krivkovich, Weddle, Price and Sohoni, “AI is changing work. Now it has to change the organization,” McKinsey Quarterly, October 5, 2026.
McKinsey & Company, “The state of AI in 2026: On the road to ROI,” August 25, 2026. Online survey of 1,719 respondents in 97 countries, May 4 to June 8, 2026.
IDC, “The Verification Tax: The Emerging Economics of AI in Finance,” white paper sponsored by Sage, June 2026. Survey of 2,275 senior finance decision makers in North America and EMEA, February 2026, via Sage press release, July 6, 2026.
MIT NANDA, The GenAI Divide: State of AI in Business 2025, July 2025. Review of 300+ public AI deployments, 52 interviews and a survey of 153 leaders, via Virtualization Review, August 19, 2025.
Our core analytics and AI platform drives informed decision-making with enhanced clarity and focus, and rapidly unlocks enterprise value
Core Features:
Integrate data from multiple source systems effortlessly
Critical for breaking down silos and creating a unified view
A secure, centralized cloud hub for all your data insights
Foundational for reliable decision-making and enterprise-wide alignment
Visualize complex data through easy-to-understand dashboards
Empowers leaders with actionable insights
Reverse ETL capabilities enriches your data and ensures your data flows exactly where it is needed for function teams to act on
Enables real-time, action-oriented data flow
BS International Business | American University of Paris
BS Computer Science | American University of Paris
Seasoned DevOps and infrastructure engineer with expertise in AWS, Kubernetes, and Terraform; led cloud migrations and scalable infrastructure projects at Sfara, FanDuel, and Kickstarter.
With over 15 years of experience in small and medium-sized startups, Scott is a seasoned expert in designing, optimizing, and maintaining robust, scalable, and secure infrastructure. He specializes in automation and embedding security from the ground up, consistently delivering reliable systems tailored to meet dynamic business requirements.
Prior to joining Teragonia, Scott made a significant impact at Sfara, where he built the company’s entire infrastructure from scratch. He engineered systems capable of supporting hundreds of thousands of users with seamless scalability, implemented automated development pipelines, and introduced observability tools to monitor and manage resources effectively. Additionally, Scott led the infrastructure team in achieving ISO27001 security certification, ensuring security was integrated into every aspect of the system and transforming it into a critical asset for business-to-business operations.
Beyond his technical expertise, Scott has a proven track record of managing and mentoring high-performing teams. As a Senior DevOps Engineer at FanDuel, he gained invaluable experience in scaling infrastructure and optimizing resources to support millions of daily users, aligning technological capabilities with organizational goals.
Master’s in Human Resources | University of Illinois
Bachelor’s in Management | Loyola University
Former Financial and Operations Manager at Houlihan Lokey, Golin Harris, and MSL Group.
Jack is a highly driven, cross functional professional with extensive experience in operations and administration.
Prior to joining Teragonia, Jack held financial and facilities management roles for Houlihan Lokey, MSL Group/Publicis, and Golin Harris in which managed and created processes and trainings for multiple functional areas ensuring operational and administrative procedures were well planned, efficient, cost-effective, and aligned with business objectives while ensuring initiatives, internal events as well as client events propelled employee and client engagement.
Jack holds undergraduate degrees from University of Illinois and Loyola University Chicago and has completed graduate certificates in Business Administration, Strategic Human Resources, and Operations at Cornell, CUNY-Buffalo, and University of Illinois and is in the process of completing a Master’s in Human Resources at Loyola University Chicago’s Quinlan School of Business.
MS Analytics | Georgia Institute of Technology
BS Management Information Systems | Oklahoma State University
Former analytics engineer at Cyderes and ConocoPhillips with a Master’s in Analytics from Georgia Institute of Technology and a Bachelor’s in Management Information Systems from Oklahoma State University
Mason is an Analytics Engineer with deep experience in data analytics, business intelligence, machine learning, and cybersecurity. He brings a proven track record of leading analytics engagements spanning architecture, insights, visualizations, and delivery.
Before joining Teragonia, Mason was a Senior Analytics Engineer at Cybersecurity MSSP CYDERES where he built a scalable, standardized, and secure analytics architecture for over 300 clients across many industries and consulted with them to deliver insights through bespoke data driven solutions. In addition, he managed the data delivery of the insight platform leveraged by the Security Operations Center to respond to incidents in a timely and effective manner.
Prior to joining CYDERES, Mason worked in ConocoPhillips’ Analytics and Innovation Center of Excellence holding varied roles within the Data Analytics organization from Data Engineering, to Business Intelligence, and Data Science. He delivered robust data solutions in all operating units for various functions including Engineering and Production, Finance, IT, and more. Including projects to standardize cost and production data across operating units.
Mason started his career at The Williams Companies in cybersecurity and transitioned to cybersecurity at ConocoPhillips where he found his passion for Data Analytics through SIEM management, detection engineering, and threat intelligence.
Bachelor’s in Finance & Accounting | Georgetown University
Former analytics engineer at Houlihan Lokey and financial analytics at JP Morgan Chase with a Bachelor’s in Finance & Accounting at Georgetown University
Grace is a seasoned analytics engineer with specialized expertise in crafting and implementing analytics solutions that drive agile, informed executive decisions in M&A and value creation for private equity-backed companies.
Before joining Teragonia, Grace was a part of the data science and business analytics team at Houlihan Lokey. She has excelled in harmonizing, enriching, and analyzing data from diverse sources, providing key insights that enabled private equity investors and portfolio company executives to make rapid, data-driven decisions across the investment lifecycle. She has developed novel analytics solutions, including deal sourcing and evaluation tools for platform investments that employ a buy-and-build or de novo growth strategy, as well as post-close value creation and KPI reporting tools for operators and management teams.
Grace has also worked at JPMorgan Chase & Co. in the Global Finance and Business Management rotational program, where she built analytics solutions to evaluate banker attrition and KPI reporting within the Global Private Bank.
