Building a scalable conversation response framework (CRF)
What happens when a chatbot responds to the user's desired outcome, not just to the topic it detects?
You get intuitive conversations that retain your user's trust while preventing them from repeating their query or becoming increasingly frustrated.
The problem
With declining CSAT and NPS scores, increasing nominal 'good-will payments', compounding pressure from challenger 'neo' banks and app-based trading platforms, the organisation launched a digital transformation.
This transformation included redesigning the chatbot experience. As of July 2026, it had a 30% resolution rate. The conversation-flow was linear, contained and didn't often correctly match the utterance with the topic. Responses were pre-canned, making responses robotic, impersonal and usually irrelevant.
Scroll for more
Context
Generally speaking, the most commonly cited reason for failed chatbot experiences is the bot's inability to detect a user's intent. While tech (AI) has rapidly advanced over recent years, this doesn't resolve the issue of effectively detecting a user's intent and responding intelligently, consistently.
Identifying the 'why' behind client contact
The organisation's commitment to client service is a consistently-cited reason why clients stay. When staff handle client queries, clients are left with a positive impression.
Conversely, NPS, CSAT, and Trustpilot reviews all point to a similar dissatisfaction with the organisation's digital experience, with clients consistently flagging it as a priority area for improvement.
Before starting any redesign of the conversational experience, I wanted to ensure I fully understood my users - including their real needs and expectations. I began by reviewing a range of qual and quant data, ranging from real chatbot transcripts to contact centre performance metrics.
I found that over 90% instances of contact (between April 2025 and 2026), clients sought at least one the following 4 outcomes:
To resolve a technical or product-based problem
Gain an understanding of a financial topic or product feature
Be supported to complete an action or take next steps
Get an update or find out the latest progress of something
Combined with previous research into our clients (including documented archetypes, needs and priorities) this insight informed the grounding structure the conversational response framework (CRF).
The CRF reads every client query across 3 layers:
Topic of conversation
Intent of contact
Client's conversation goal
Using the detail within each layer, the chatbot adjusts its routing or 'mode' and subsequent responses - going beyond routing on the detected topic.
Client feedback consistently praises contact-centre staff for their friendliness and their supportive approach. Usually, those characteristics come from the colleague 'reading' the room, being empathetic, acknowledging surrounding detail of a request and simply being human.
While my ambition was never to replicate the human-touch in an AI Chatbot (also, ethically and regulatorily risky) the CRF encodes a similar instinct within the chatbot. This means that responses can go beyond accuracy, and actually respond to the client's need in a way that's genuinely helpful.
Using 'why' to shape conversation standards
For both the 'intent' and the 'conversation goal', I developed and defined standards to inform each response for common contact scenarios:
Testing and iterating
As of August 2026, we're currently about to start real-user testing.
However, I've been able to continuously stress test the CRF against real transcripts and queries, helping me identify opportunities to improve, knowledge and framework gaps - then dynamically iterate as a result of that process.
The outcome
The CRF marks a significant departure from the existing (and anticipated) conversational design approach. Like many other organisations, the current structure of knowledge, conversation flows and website was based on topics, not actual actions or user needs. As a result, current digital responses were missing element for delivering quality experiences, understanding the why motivating the contact.
I tested the performance against real queries from transcripts, to see how the conversation would perform against the CRF.
From:
"the AI understands the question."
To:
"the AI understands the support the client is seeking."
I designed
A scalable framework that supports British investors to self-serve within a human, empathetic and intuitive conversational interface.
User pains
Current PVA chatbot misunderstands simple queries
Looping, unhelpful chatbot responses
Limited support available during non-working hours (human contact centre operates 9am-5pm)
Business pains
Current chatbot ~30% resolution rate
Declining NPS and CSAT scores - dissatisfaction with digital experience cited frequently
Increasing demand on colleague-manned contact centres
Impact (metrics)
Projected 78% increase conversation resolution rate
Better implementation
Released
Scheduled Q4 2026
Disclaimer
I've structured case studies to remain compliant with IP and confidentiality obligations of my former employers. I've expressed any metrics as percentages or directional improvements. Any internal documents have been recreated and I have not included any names of of stakeholders, figures or other proprietary system details.
Password protected
To comply with NDAs and confidentiality agreements, a password is required to view my previous work.
Email alisha.oni@googlemail.com if you're struggling to access