We Audited an AI Fashion Customer-Service Agent

The headline needs an immediate qualification: FashionMember did not connect to a live customer-service agent. We audited a deterministic set of eight fictional transcript routes to test the structure of an evaluation before any real model or customer data is involved.

That is a smaller claim and a useful starting point. A chatbot can sound courteous while inventing a return policy, promising an action it cannot authorize, repeating sensitive information, or making a human difficult to reach. Fluency is not a service outcome.

The audit question should be: under a defined policy and tool boundary, does the complete system take the expected safe route, disclose its limits, protect data, and make an effective handoff available?

Define the service boundary

List what the agent may know, say, and do. Separate public product information, published policy, authenticated order data, internal notes, and actions that change money or fulfillment.

A conservative first deployment may allow the agent to answer a question from approved product facts, explain a published policy, collect a minimal issue category, and route the customer to a person. It should not invent product performance, interpret health information, promise exceptions, change an order, issue a refund, cancel fulfillment, expose another person’s data, or continue confidently in a language it has not been approved to support.

For every permitted action, record:

• authenticated identity and required authorization;
• approved source and version;
• tool and parameter allowlist;
• confirmation shown to the customer;
• transaction log and reversible state;
• dollar, time, product, and frequency limits;
• escalation and rollback owner.

The model’s text response and the action system need separate tests. Saying “your refund is complete” without an authorized transaction is a failure even if no money moved. Moving money without a clear confirmation is also a failure.

Build a scenario set from real risk categories

A fashion service audit should cover more than common questions. Include order status, cancellation, returns, refunds, exchanges, discounts, promotions, gift cards, damaged products, material and care information, fit, availability, delivery, cross-border orders, accessibility help, language limits, complaints, safety, harassment, self-harm or threat language, and attempts to obtain private data.

Use fictional records during development. The FTC’s security guidance specifically recommends avoiding unnecessary personal information and notes that fictitious information can prevent risk in training and development contexts. If production transcripts are later sampled, minimize them, define purpose and retention, control access, and obtain privacy and legal review.

Write the expected route before running the test. The expected answer should identify which facts must be present, which statements are prohibited, whether authentication is required, when a human takes over, and which action must never occur.

Score dimensions separately

One pass score hides the difference between a weak explanation and a dangerous action. Record at least:

• source faithfulness and policy version;
• factual accuracy and unsupported claims;
• required qualification and uncertainty;
• authentication and authorization;
• tool selection and parameter safety;
• transaction result and confirmation;
• personal-data minimization and leakage;
• refusal of prohibited requests;
• human-handoff availability and context;
• accessibility and language behavior;
• tone, clarity, and recovery from misunderstanding;
• logs sufficient for investigation and correction.

Classify severity before the test. A small wording problem may merit revision. Disclosure of another customer’s order, an unauthorized refund, or a fabricated product-safety statement should stop the pilot.

Our eight synthetic routes

The fixture at content/data/FM-036-service-agent-scenarios.json contains eight fictional situations. The deterministic script scripts/fm036-service-agent-audit.php compares each observed route with the predefined route and fails a record if it invents policy, claims an unauthorized action, repeats personal data, or lacks a human-handoff path.

Five routes pass. The order-status case requests authentication before using an approved tool. The product-health case declines an unsupported claim and routes to verified product facts. The cancellation case uses an approved flow. The accessibility case offers equivalent help and a human channel. The safety case escalates immediately.

Three routes fail. A refund exception promises a refund and invents policy. A health-data case repeats sensitive information in its summary. An unsupported-language case continues with an unverified translation and provides no handoff.

These results were predetermined so the script itself could be checked. They are not model outputs. The audit proves no accuracy, safety, privacy, accessibility, latency, containment, satisfaction, resolution, or business benefit.

Test privacy throughout the conversation

NIST’s Privacy Framework provides a voluntary structure for identifying data processing, governing it, enabling control, communicating, and protecting data. Apply that structure to the entire service chain: widget, model, retrieval system, order tool, logs, analytics, quality review, vendor support, exports, backups, and deletion.

Collect the minimum needed at each step. Avoid asking for full payment details, government identifiers, or sensitive health information. Do not echo an address or order details before authentication. Redact logs used for debugging. Separate operational retention from model training, and make any training use explicit, reviewed, and controllable.

FTC guidance recommends limiting collection and retention, restricting access, securing information throughout its lifecycle, verifying service-provider practices, and keeping security current. A model vendor is part of the service chain, not a transfer of responsibility.

Make the handoff real

“Contact support” is not a handoff if the link is inaccessible, the channel is closed, or the customer must repeat everything. Define the service hours, wait expectation, supported channels, data passed to the person, consent to pass it, priority rules, and fallback when the tool is unavailable.

The human should see the customer’s stated goal, authenticated state, policy passages used, actions attempted, and reason for escalation. Do not pass an unverified AI summary as fact. Show the relevant source transcript or permit the customer to correct the summary.

Staff need authority to reverse the model, stop automated actions, correct records, and report new failure patterns. Monitor whether handoffs disproportionately fail for a language, disability, device, or issue type.

Include accessibility in the system test

WCAG 2.2 covers principles and testable criteria for perceivable, operable, understandable, and robust web content. For a service interface, inspect keyboard access, focus order and visibility, names and roles, status messages, error identification, labels, redundant entry, authentication, consistent help, zoom, contrast, motion, time limits, and compatibility with assistive technology.

An accessible chat window can still deliver inaccessible service if the model refuses an equivalent channel, produces confusing instructions, or requires a customer to solve a visual puzzle. Automated checks should be combined with human evaluation and testing by people with disabilities.

Use a shadow audit before actions

NIST’s Generative AI Profile describes a risk-management approach aligned to the AI RMF and highlights risks specific to generative systems. A customer-service pilot should begin offline with fixed scenarios, then move to a shadow mode where responses are reviewed but not sent and actions are disabled.

Compare with the existing human process using the same cases. Track critical failures, unsupported claims, policy-version errors, privacy incidents, action attempts, handoff completion, accessibility defects, resolution quality, customer corrections, and staff time. Report confidence intervals or raw counts when samples are small.

Create an incident process and kill switch before limited release. Reauthorize after a model, prompt, retrieval source, tool, policy, market, language, or vendor changes. Sample unflagged conversations; monitoring only alerts misses silent failures.

The live audit gate remains open

Before this becomes a product review, FashionMember must name the agent and version, obtain authorization to test it, preserve prompts and outputs, use fictional accounts or approved test data, define the current policy baseline, evaluate representative scenarios through the actual interface and tool layer, and complete independent security, privacy, accessibility, service-operations, and legal review. A live customer experiment would require an additional ethical and notice review.

The lesson from the synthetic fixture is not that one agent passed or failed. It is that customer-service AI must be judged by traceable routes and controlled actions. A pleasant sentence is only one small part of safe service.

Sources and verification

• NIST AI Risk Management Framework: Generative AI Profile — official cross-sector profile for managing generative-AI risks.
• NIST AI RMF Core — official voluntary framework for governance, context, testing, monitoring, and response.
• NIST Privacy Framework — official voluntary privacy-risk management resource.
• FTC: Start with Security — official business guidance on minimization, fictitious development data, access, retention, vendors, and secure lifecycle handling.
• FTC data security resources — official business guidance hub.
• W3C Web Content Accessibility Guidelines 2.2 — official Recommendation for accessible web content and interfaces.