Building a scalable conversation response framework (CRF)

What happens when a chatbot responds to the user's desired outcome, not just to the topic it detects?

You get intuitive conversations that retain your user's trust while preventing them from repeating their query or becoming increasingly frustrated.

The problem

With declining CSAT and NPS scores, increasing nominal 'good-will payments', compounding pressure from challenger 'neo' banks and app-based trading platforms, the organisation launched a digital transformation.

This transformation included redesigning the chatbot experience. As of July 2026, it had a 30% resolution rate. The conversation-flow was linear, contained and didn't often correctly match the utterance with the topic. Responses were pre-canned, making responses robotic, impersonal and usually irrelevant.

The problem

With declining CSAT and NPS scores, increasing nominal 'good-will payments', compounding pressure from challenger 'neo' banks and app-based trading platforms, the organisation launched a digital transformation.

This transformation included redesigning the chatbot experience. As of July 2026, it had a 30% resolution rate. The conversation-flow was linear, contained and didn't often correctly match the utterance with the topic. Responses were pre-canned, making responses robotic, impersonal and usually irrelevant.

Scroll for more

Context

Generally speaking, the most commonly cited reason for failed chatbot experiences is the bot's inability to detect a user's intent. While tech (AI) has rapidly advanced over recent years, this doesn't resolve the issue of effectively detecting a user's intent and responding intelligently, consistently.

Context

Generally speaking, the most commonly cited reason for failed chatbot experiences is the bot's inability to detect a user's intent. While tech (AI) has rapidly advanced over recent years, this doesn't resolve the issue of effectively detecting a user's intent and responding intelligently, consistently.

Identifying the 'why' behind client contact

The organisation's commitment to client service is a consistently-cited reason why clients stay. When staff handle client queries, clients are left with a positive impression.

Conversely, NPS, CSAT, and Trustpilot reviews all point to a similar dissatisfaction with the organisation's digital experience, with clients consistently flagging it as a priority area for improvement.

Before starting any redesign of the conversational experience, I wanted to ensure I fully understood my users - including their real needs and expectations. I began by reviewing a range of qual and quant data, ranging from real chatbot transcripts to contact centre performance metrics.

I found that over 90% instances of contact (between April 2025 and 2026), clients sought at least one the following 4 outcomes:

  1. To resolve a technical or product-based problem

  2. Gain an understanding of a financial topic or product feature

  3. Be supported to complete an action or take next steps

  4. Get an update or find out the latest progress of something

Identifying the 'why' behind client contact

The organisation's commitment to client service is a consistently-cited reason why clients stay. When staff handle client queries, clients are left with a positive impression.

Conversely, NPS, CSAT, and Trustpilot reviews all point to a similar dissatisfaction with the organisation's digital experience, with clients consistently flagging it as a priority area for improvement.

Before starting any redesign of the conversational experience, I wanted to ensure I fully understood my users - including their real needs and expectations. I began by reviewing a range of qual and quant data, ranging from real chatbot transcripts to contact centre performance metrics.

I found that over 90% instances of contact (between April 2025 and 2026), clients sought at least one the following 4 outcomes:

  1. To resolve a technical or product-based problem

  2. Gain an understanding of a financial topic or product feature

  3. Be supported to complete an action or take next steps

  4. Get an update or find out the latest progress of something

Combined with previous research into our clients (including documented archetypes, needs and priorities) this insight informed the grounding structure the conversational response framework (CRF).

The CRF reads every client query across 3 layers:

  1. Topic of conversation

  2. Intent of contact

  3. Client's conversation goal

Using the detail within each layer, the chatbot adjusts its routing or 'mode' and subsequent responses - going beyond routing on the detected topic.

Client feedback consistently praises contact-centre staff for their friendliness and their supportive approach. Usually, those characteristics come from the colleague 'reading' the room, being empathetic, acknowledging surrounding detail of a request and simply being human.

While my ambition was never to replicate the human-touch in an AI Chatbot (also, ethically and regulatorily risky) the CRF encodes a similar instinct within the chatbot. This means that responses can go beyond accuracy, and actually respond to the client's need in a way that's genuinely helpful.

Using 'why' to shape conversation standards

For both the 'intent' and the 'conversation goal', I developed and defined standards to inform each response for common contact scenarios:

Client feedback consistently praises contact-centre staff for their friendliness and their supportive approach. Usually, those characteristics come from the colleague 'reading' the room, being empathetic, acknowledging surrounding detail of a request and simply being human.

While my ambition was never to replicate the human-touch in an AI Chatbot (also, ethically and regulatorily risky) the CRF encodes a similar instinct within the chatbot. This means that responses can go beyond accuracy, and actually respond to the client's need in a way that's genuinely helpful.

Using 'why' to shape conversation standards

For both the 'intent' and the 'conversation goal', I developed and defined standards to inform each response for common contact scenarios:

Testing and iterating

Testing and iterating

As of August 2026, we're currently about to start real-user testing.

However, I've been able to continuously stress test the CRF against real transcripts and queries, helping me identify opportunities to improve, knowledge and framework gaps - then dynamically iterate as a result of that process.


As of August 2026, we're currently about to start real-user testing.

However, I've been able to continuously stress test the CRF against real transcripts and queries, helping me identify opportunities to improve, knowledge and framework gaps - then dynamically iterate as a result of that process.


  • A

  • A

  • A

  • B

  • C

The outcome

The CRF marks a significant departure from the existing (and anticipated) conversational design approach. Like many other organisations, the current structure of knowledge, conversation flows and website was based on topics, not actual actions or user needs. As a result, current digital responses were missing element for delivering quality experiences, understanding the why motivating the contact.

I tested the performance against real queries from transcripts, to see how the conversation would perform against the CRF.

The outcome

The CRF marks a significant departure from the existing (and anticipated) conversational design approach. Like many other organisations, the current structure of knowledge, conversation flows and website was based on topics, not actual actions or user needs. As a result, current digital responses were missing element for delivering quality experiences, understanding the why motivating the contact.

I tested the performance against real queries from transcripts, to see how the conversation would perform against the CRF.

From:
"the AI understands the question."
To:
"the AI understands the support the client is seeking."

I designed

A scalable framework that supports British investors to self-serve within a human, empathetic and intuitive conversational interface.

User pains

  • Current PVA chatbot misunderstands simple queries

  • Looping, unhelpful chatbot responses

  • Limited support available during non-working hours (human contact centre operates 9am-5pm)

Business pains

  • Current chatbot ~30% resolution rate

  • Declining NPS and CSAT scores - dissatisfaction with digital experience cited frequently

  • Increasing demand on colleague-manned contact centres

Impact (metrics)

  • Projected 78% increase conversation resolution rate

  • Better implementation

Released

Scheduled Q4 2026

Disclaimer

I've structured case studies to remain compliant with IP and confidentiality obligations of my former employers. I've expressed any metrics as percentages or directional improvements. Any internal documents have been recreated and I have not included any names of of stakeholders, figures or other proprietary system details.

Next project

Building a cohesive product voice

What's the password?

To comply with NDAs and confidentiality agreements, you'll need my password to view past projects.

Email alisha.oni@googlemail.com if you're struggling to access