ECOMMERCE AI CHATBOT EVALUATION CHECKLIST
By Roman Mohren | FullSession | 19 October 2026

Purpose: compare required shopper jobs in the actual proposed configuration.
This is a blank evaluation tool, not a product test result. The retail cases below
are fictional expectations. Replace all example data and policies before testing.

Store/platform: ____________________   Vendor/configuration: ____________________
Evaluator/date: ____________________   Channels/markets/languages: _______________
Required scope: buying answers / order lookup / action / human continuity

1. STATE THE JOB AND ITS SOURCE
[ ] Write one recurring buying question and its approved product source.
[ ] Identify current stock/variant sources and the applicable refresh behavior.
[ ] Write one recurring order-help request and one exception your people handle.
[ ] Specify what the action may change, who permits it and what proves completion.
[ ] Identify the receiving human destination and unavailable-team response.

2. USE THE SAME CASES FOR EVERY VENDOR
Fictional normal case: an outdoor jacket shopper asks which jacket fits over fleece.
Expected: relevant approved fit facts; clarification when preference is missing.
Fictional order case: authorized customer wants a new delivery address.
Invented policy: unfulfilled eligible order may be changed after confirmation;
already shipped order needs a person. These are not any vendor's default policy.
Expected: correct identity/order, eligibility check, required confirmation,
supported change and confirmed resulting address; shipped exception handed off.

Case                              Observed behavior/evidence        Issue owner
Buying question + missing fact    ____________________________      ____________
Authorized current order lookup   ____________________________      ____________
Wrong identity/order access       ____________________________      ____________
Eligible supported address change ____________________________      ____________
Already shipped exception         ____________________________      ____________
Action timeout or uncertain state ____________________________      ____________
Repeated request after timeout    ____________________________      ____________
Human team unavailable            ____________________________      ____________

For each case record: expected answer/state; actual response; source/version;
identity scope; attempted action; resulting state; known/unknown result;
handoff contents/destination; configuration; observation date; evidence location.
Do not mark an empty observation as a pass. Distinguish prepared, attempted,
confirmed complete, confirmed failure and unknown. Check state before retrying.

3. ESTABLISH CONFIGURATION AND OPERATING FIT
[ ] Verify the named store connection, source format and action in the actual setup.
[ ] Confirm identity, access, approval and platform/account prerequisites.
[ ] Confirm source-update/removal behavior, catalog variants and market conditions.
[ ] Test the relevant web/email/social channels rather than generalizing from one.
[ ] Check what the receiving person sees and who owns follow-up.
[ ] Separate supported scope from dependencies with named owners and effort.

4. GET A COMPLETE QUOTE
Base platform / AI meter / allowance or minimum / seat terms / channels / actions /
optional services / recording / third-party subscriptions / taxes and overages.
For Max: Desk, shared inbox, human chat and supported integrations are included;
no seat fees. Verify current offer at https://www.fullsession.io/pricing?solution=desk.
No comparative billing-unit conversion is built into this checklist.

5. DECIDE
Essential case results: ________________________
Missing dependencies and owners: _______________
Required corrections before use: _______________
Selected scope and reason: _____________________
Keep serious access failures and false completion visible; a useful answer in
another case does not cancel them. Review real outcomes after a defined launch.

NEXT STEP
Bring one product/buying question and one order-help exception to a Max walkthrough:
https://www.fullsession.io/book-a-demo?solution=support-desk