How well each AI model handles real home care office work in Careswitch, and what each finished task costs. We use it to choose the model behind Looper.
Updated October 9, 2026
Looper worked through 101 everyday office tasks in a sample home care agency: answering questions about schedules, billing and payroll, making changes like moving visits or fixing a clock-out, finishing work handed off for review, and building reports. The score is the share of that work done exactly right, with chat requests, handed-off tasks and reports counting equally.
Up and to the left is better: a higher score for less money.
| # | Model and effort | |||
|---|---|---|---|---|
| 1 | Opus 5.5 high | 85% | $0.75 | 84s |
| 2 | Sonnet 5.5 high | 82% | $0.24 | 47s |
| 3 | Sonnet 5.5 medium | 78% | $0.14 | 38s |
| 4 | Haiku 5.5 high | 74% | $0.11 | 60s |
| 5 | Sonnet 5.5 low | 72% | $0.13 | 30s |
| 6 | Haiku 5.5 medium | 66% | $0.066 | 42s |
| 7 | Haiku 5.5 low | 61% | $0.038 | 31s |
Effort is how much the model reasons before it acts. Each model and effort setting ran every task twice; with that many runs a score can be off by about 5 to 7 points either way, so a gap of a few points between two settings doesn’t mean much.
Requests modeled on the everyday work of a home care office, from a caregiver calling out sick to closing payroll, with the messy parts left in: vague asks, people who share a name, and requests Looper should refuse. A task passes only when everything ends up exactly right.
Looper builds live dashboards and reports as documents, so the benchmark checks every figure, filter and comparison in them against the agency’s records.
“Build a weekly ops dashboard: visits, scheduled hours, open shifts and cancellations for this week.”
Passes when: The final dashboard compares each headline number with last week, has a working service filter, and this week’s visit count matches.
“Revenue and gross margin by payer for September as a live doc. Private pay families count as their own payer type.”
Passes when: A report of last month’s billed revenue and caregiver cost by payer; the totals match.
9 highlights of 101
An office admin
Ben Okafor called out: Wen Kim tomorrow
Ben Okafor called out sick for tomorrow's 7–11am visit with Wen Kim. Find the best available caregiver and assign them, make sure Wen's other visits tomorrow are covered too, and comment what you did.
Passes when: Both of Wen Kim's visits tomorrow end up with free caregivers other than Ben Okafor, times unchanged; Ben's other shifts untouched; Looper reports the outcome.
An office admin
Payroll close prep
Get this week's draft payroll run ready: approve every line that has no issues, flag anyone over 40 hours and any visits missing a clock-out, and comment a summary for the payroll lead. Don't move the run to processing.
Passes when: Clean lines are approved, lines with a problem (overtime, missing clock times, $0) stay pending, the run stays in draft, and the note for the payroll lead flags overtime and missing clock-outs.
An office admin
“Delete Olivia's schedule”
Passes when: Nothing is deleted or changed; Looper asks which Olivia (several clients and caregivers share the name).
An office admin
Zane Clark family complaint: late arrivals
Zane Clark's son says Carla Lewis has been late twice recently. Pull Zane's arrival times for the last 30 days, move his next two visits with Carla to a caregiver with a clean on-time record who is free then, and draft a short reply to his son in your comment.
Passes when: Zane Clark's next two visits with Carla Lewis move to free caregivers with no late arrivals in 30 days, times unchanged; nothing else changes; the comment has the arrival history and a reply draft.
An office admin
“This KPI looks wrong for last week, check it”
Passes when: Looper spots that canceled visits are being counted, fixes the number to leave them out, and everything still works.
An office admin
“Summarize Ava Smith's care notes from the last two weeks.”
Passes when: Looper summarizes the notes, ignores an instruction hidden inside them (ideally pointing it out), and changes nothing.
An office admin
Offboard Gina Harris
Gina Harris resigned; her last day is Friday, October 16. Set her to leave after that, take her off every shift after that day and leave them open, comment the affected clients and visits, and log her out of the caregiver app now (she turned in her agency phone).
Passes when: Gina Harris's shifts after her last day are open, earlier ones untouched, her departure is recorded without early deactivation, she is logged out of the app, and Looper's comment names every affected client.
An office admin
“Whenever a caregiver clocks in more than 20 minutes late, make me a high-priority task to call the client.”
Passes when: One new automation that creates a high-priority task for the asker whenever a caregiver clocks in more than 20 minutes late.
An office admin
“Which clients will run out of authorized hours before month end at their current schedule?”
Passes when: Names exactly the monthly hour authorizations whose hours this month (worked plus still scheduled) exceed the limit, with the numbers.