The Challenge

A leading global financial institution with a team of analysts and decision-makers faced a critical challenge: how to ensure generative AI systems could be trusted in high-stakes workflows.

Analysts relied on large language models (LLMs) to accelerate tasks such as financial analysis, reporting, and compliance checks. However, these systems frequently produced hallucinations - outputs that looked correct but were factually wrong. Even minor errors in earnings figures, guidance numbers, or margin calculations could cascade into multi-million-dollar risks.

The institution needed a solution that could:

• Continuously verify AI outputs and autocorrect mistakes in real time.

• Flag uncertain answers for human review before they reached decision-makers.

• Deliver measurable gains in accuracy, consistency, and user trust without slowing workflows.

The Solution

The enterprise partnered with Pegasi AI to pilot a corrective layer for LLMs. Pegasi’s system was designed to validate model outputs against retrieved evidence, fix errors automatically, and quantify uncertainty in responses.

Pegasi’s role included:

• System Architecture & POCs: Designing and implementing the first prototypes for correction pipelines.

• Autocorrection Models: Iterating on fine-tuned models capable of span-level error detection and repair.

• Secure Deployment: Assisting with integration into the client’s existing AI stack under enterprise security requirements.

Key Features

• Real-Time Autocorrection: Automatically lifts accuracy of LLM responses without human intervention.

• Uncertainty Awareness: Flags low-confidence outputs for analyst review, preventing silent failures.

• Context-Aware Validation: Detects whether the necessary ground truth exists in the retrieved context before scoring an answer.

• Enterprise-Grade Infrastructure: Built for compliance, auditability, and scale in regulated environments.

The Impact

The pilot demonstrated that Pegasi’s corrective layer significantly improved model reliability:

• Overall accuracy lifted from ~67% to 91%+ across a dataset of 150+ financial QA tasks.

• On subsets where the correct context was retrieved, accuracy improved from 76% to 98%, with potential to reach 100% once data quality issues were resolved.

• In less than a week, internal teams consisting of PhDs and analysts validated that the system consistently outperformed baseline LLM outputs, reducing analyst review time and boosting trust in AI-assisted workflows.

Error analysis further showed that residual mistakes were often due to data quality issues (e.g., conflicting evidence, mislabeled ground truth) rather than Pegasi’s system itself.

Looking Ahead

Following the pilot’s success, the institution is exploring:

• Expansion into compliance, legal, and reporting workflows.

• Deployment of Pegasi models, which offer improved context retrieval and uncertainty quantification.

• Broader rollout across teams to accelerate adoption while reducing operational risk.

Pegasi is helping leading enterprises build trust in AI by delivering real-time correction, higher accuracy, and measurable business impact.