Deploying In-House LLM Agents for Customer Support
Customer support is where a lot of AI ambition meets reality. Demos are easy; a support agent that customers actually trust, that resolves issues, and that does not invent answers is hard. At Good Glamm Group we built in-house AI-powered (LLM) customer service agents and put them into production.
Why in-house
Off-the-shelf bots are quick to stand up but slow to trust. They do not know your catalogue, your policies, or your edge cases, and you cannot tune their behaviour when they get something wrong. Building in-house meant we controlled the knowledge the agents drew on, the guardrails on what they could say, and the escalation path to a human.
The outcome
- NPS lifted by 14% — customers got faster, more consistent resolutions.
- Faster service turnaround without a proportional increase in support cost.
What I would tell anyone shipping LLMs to production
- Ground the model. An LLM answering from your actual policies and order data is useful; one answering from its training prior is a liability.
- Design the handoff. The goal is not to remove humans, it is to let humans spend their time on the cases that need them.
- Measure the business metric, not the model metric. We cared about NPS and turnaround, not token-level scores.
Generative AI is most valuable when it is pointed at a high-frequency, high-friction workflow and held to a real business outcome. Support is one of the clearest examples.