AT&T
Governed enterprise genAI assistant: an LLM connected to internal knowledge, open to tens of thousands of employees and called at scale by automated systems
AT&T deployed Ask AT&T, a governed internal genAI assistant (Azure OpenAI) used by more than 100,000 employees across 71 solutions, processing about 9 billion tokens per day and reducing customer care resolution time by 33%.
Key points
- Governed internal genAI assistant (Ask AT&T) for employees.
- Azure OpenAI LLM plus RAG on internal knowledge, called via API.
- Care resolution time -33%, 71 solutions, ~9 billion tokens per day.
- Evidence level B, confirmed active status.
Objective
Put a reliable AI assistant in employees' hands to shorten customer resolution and industrialize productivity gains, under governance.
The deployment
Ask AT&T is AT&T's internal genAI assistant, launched in June 2023. It gives access to an LLM connected to company knowledge to summarize documents, query contracts, write code, analyze the network, or answer general questions. Usage grew fast: more than 80,000 employees had access by summer 2024, and more than 100,000 by the end of 2025. AT&T says it deployed 71 distinct genAI solutions and reduced customer care resolution time by 33 percent. The platform processes about 9 billion tokens per day, most coming from internal systems calling it via API rather than from humans. AT&T links these solutions to annual savings in the hundreds of millions, as part of a group-wide cost reduction program.
Results Proof B
Operational figures confirmed by a Microsoft customer story with a named AT&T spokesperson (71 solutions, 100,000 employees, 9 billion tokens/day, -33 percent care resolution time) and picked up by the specialist press. Vendor source plus press: solid on scale and adoption; the savings remain aggregated at group level, not isolated per case.
How it works
Documented architectureThe stack in detail
- llm Azure OpenAI OpenAI models served via Azure, the LLM component of Ask AT&T; the exact model versions are not detailed publicly.
- infra Azure Cosmos DB Azure database supporting the platform and its high-volume use cases.
- plateforme Ask AT&T (plateforme interne) Governed foundation built by AT&T: RAG on internal knowledge, access control, logging, API for internal systems; 71 solutions built on it.
- infra Microsoft Azure Cloud infrastructure for the setup, operated within the AT&T-Microsoft partnership.
How it runs, concretely
For ops teams-
1Open a governed foundation Data science / AI team
A single platform gives access to the LLM with access control, logging, and usage rules, rather than unmanaged tools per team.
-
2Let the business units build Business units + AI
Each team connects its use case (care, code, contracts, network) to the foundation via RAG. 71 solutions were built this way.
-
3Industrialize via API Internal systems / AI
Internal systems call the assistant at scale (most of the 9 billion tokens per day), not only humans at their desks.
-
4Measure and govern continuously AI team + management
Tracking of care resolution time, adoption, and savings; review of safeguards as usage grows.
Governance: each use case goes through a control before production. Without that filter, an assistant open to 100,000 people propagates false answers or data leaks at scale.
How your customers perceive this type of use
Sourced studiesLes consommateurs n'acceptent pas les chatbots par defaut : 64% prefereraient que les entreprises n'utilisent pas d'IA dans leur service client (Gartner, 2024) et pres d'un utilisateur sur cinq du service client par IA n'en retire aucun benefice (Qualtrics, 2025). L'acceptation se construit sur trois conditions mesurees par Salesforce : savoir qu'on parle a une IA, pouvoir escalader vers un humain, comprendre la logique de l'agent.
Acceptance conditions
- Etre informe qu'on parle a une IA et non a un humain (pres de 75% le demandent, Salesforce 2024)
- Un chemin d'escalade clair vers un agent humain (45% plus enclins a utiliser l'agent IA, Salesforce 2024)
- Une logique de l'agent clairement expliquee (44% plus enclins, Salesforce 2024)
Red lines
- Rendre l'humain injoignable : c'est la premiere inquietude des consommateurs sur l'IA dans le service client (Gartner 2024) et 50% craignent que l'IA les coupe du contact humain (Qualtrics 2025)
- Remplacer le service client par l'IA sans alternative : 53% envisageraient de partir chez un concurrent (Gartner 2024)
Sources: Salesforce 2024 · Gartner 2024 · Qualtrics 2025
How to replicate
Inference, not sourcedData prerequisites
- Accessible internal knowledge (documents, procedures, contracts, business databases)
- A single foundation with access control and logging
- Governance and compliance rules on the exposed data
Org prerequisites
- A central AI team that maintains the platform and the safeguards
- Business units authorized to build their use cases on it
- Change management / training at scale
Possible stack
- Azure OpenAI, or Vertex AI, or a market LLM in RAG
- Vector database on the internal documentation
- Governance and prompt-logging layer
The plan, step by step
- Step 1Pick a first high-volume care case and set the governance: access control, prompt logging, usage and compliance rules.Deliverable: Usage charter and pilot scope validated.
- Step 2Build the foundation (LLM + RAG on the care documentation) and put it in the hands of a pilot group of advisors.Deliverable: Pilot assistant in service, resolution time measured.
- Step 3Compare resolution time before/after, correct false answers, and harden the safeguards.Deliverable: Quantified pilot assessment and list of applied fixes.
- Step 4Open the foundation to other teams (code, contracts, network) with a review process before each case goes to production.Deliverable: 3-5 use cases in production under governance.
- Step 5Expose the assistant via API to internal systems and track adoption, volumes, and savings.Deliverable: Active internal API, usage and savings dashboard.
First step: Build a governed foundation with a first high-volume care case, measure resolution time before/after, then open building to other teams once the safeguards are proven.
The same pattern, proven in other industries
ADT
genAI customer agent
Air India
genAI customer agent (automated customer service)
Bank of America
conversational financial assistant at scale
Sources
- S1 AT&T creates digital coworkers with Azure to scale AI that works Interested party archive pending
- S2 How AT&T's HR team steers 80,000 workers in using its internal generative AI tool Secondary archive pending
- S3 AT&T's New Generative AI Tool Will Help Support Employees Primary archive pending
An error, newer info, a source?
This page lives on its accuracy. If a figure has moved, if the deployment has changed, or if you have a higher-quality source, tell us. Every sourced correction is verified before publication.