Customer Email Intent Classification

ClassificationDifficulty: HARD11 models comparedTotal cost: $0.002845

Prompt

Classify the intent of the following customer support email into exactly ONE of these categories: COMPLAINT, REFUND_REQUEST, TECHNICAL_SUPPORT, BILLING_INQUIRY, CANCELLATION, or GENERAL_INQUIRY.

Respond with only the category label and a one-sentence justification.

Email:
---
Subject: Re: Order #48821 — Still waiting

Hi,

I've now been charged twice for the same order and I've been trying to get this resolved for over two weeks. I already spoke to someone last Tuesday who said the duplicate charge would be reversed within 3–5 business days, but I just checked my bank statement and the money is still not back. At this point I'm not even sure I want the product anymore — I may just want everything canceled and a full refund. But honestly, first I just need someone to explain what's happening with my account because this is completely unacceptable.

Please respond ASAP.
— Deborah
---

Expected Output

BILLING_INQUIRY — Although the email touches on frustration (complaint), a potential cancellation, and a possible refund, Deborah's most immediate and explicit request is for an explanation of what is happening with the duplicate charge on her account, making billing inquiry the primary intent.

Model Responses

Claude Opus 4.6

Unrated
3,221ms276 tokens
BILLING_INQUIRY While the email contains elements of a complaint, refund request, and potential cancellation, the core intent is seeking an explanation for the duplicate charge and understanding what is happening with her account, making it primarily a billing inquiry.

Sign in to vote and see results

GPT-4.1

Unrated
877ms222 tokens
BILLING_INQUIRY — The customer is primarily seeking an explanation regarding duplicate charges and the status of a promised reversal.

Sign in to vote and see results

GPT-4.1 Mini

Unrated
857ms221 tokens
COMPLAINT The customer is expressing frustration about being double charged and delayed resolution, seeking an explanation and urgent response.

Sign in to vote and see results

GPT-4.1 Nano

Unrated
589ms228 tokens
CANCELLATION — The customer is expressing frustration over a double charge, unresolved issues, and the possibility of canceling the order and requesting a refund.

Sign in to vote and see results

o3

Unrated
5,935ms694 tokens
REFUND_REQUEST — The email centers on getting the duplicate charge reversed and possibly a full refund, indicating the customer’s primary aim is to recover the money paid.

Sign in to vote and see results

o3 Mini

Unrated
8,618ms1143 tokens
BILLING_INQUIRY – The customer’s primary concern is regarding an incorrect duplicate charge on their account and the lack of a refund, requesting an explanation and resolution for the billing error.

Sign in to vote and see results

o4 Mini

Unrated
3,348ms496 tokens
REFUND_REQUEST The customer was charged twice for the same order and is seeking a full refund and explanation of the duplicate charge.

Sign in to vote and see results

GPT-4o Mini

Unrated
1,688ms223 tokens$0.000045
BILLING_INQUIRY The customer is primarily seeking clarification about duplicate charges on her account, which falls under billing inquiries.

Sign in to vote and see results

Claude Haiku 4.5

Unrated
1,760ms286 tokens$0.000542
BILLING_INQUIRY Justification: While the email touches on multiple issues (duplicate charges, refund possibility, and frustration), the primary intent is to understand and resolve the billing problem—the duplicate charge and its status—which takes precedence over the secondary mention of potential cancellation.

Sign in to vote and see results

GPT-4o

Unrated
2,122ms223 tokens$0.000752
BILLING_INQUIRY The email primarily concerns being charged twice for an order and seeking an explanation of the account status.

Sign in to vote and see results

Claude Sonnet 4.6

Unrated
2,871ms278 tokens$0.001506
**COMPLAINT** Deborah is primarily expressing frustration and dissatisfaction over an unresolved duplicate charge and a failed promise of resolution, which makes the dominant intent a complaint, even though refund and cancellation desires are mentioned as secondary concerns.

Sign in to vote and see results

Cost & Performance Comparison

ModelLatencyTokensCostVerdict
Claude Opus 4.63,221ms276
GPT-4.1877ms222
GPT-4.1 Mini857ms221
GPT-4.1 Nano589ms228
o35,935ms694
o3 Mini8,618ms1,143
o4 Mini3,348ms496
GPT-4o Mini1,688ms223$0.000045
Claude Haiku 4.51,760ms286$0.000542
GPT-4o2,122ms223$0.000752
Claude Sonnet 4.62,871ms278$0.001506