Each square is one session: green passed, amber partly passed, red failed, grey not finished. Select a square to replay it, with a screenshot of every step.
Agent
T1
T2
T3
T4
T5
T6
Claude Sonnet 5.5
GPT-6.1 Sol
Gemini 4 Argon
DeepSeek V4.1 Flash
T1 I run a small marketing team and we're thinking about Asana. Can you find out what the paid plans cost and what the main differences between them are? I'd like to know if there's a free option too.
T2 What are the 'AI Teammates' Asana talks about? Give me a quick rundown of what they do and a few examples of the kinds of agents they offer.
T3 We use Slack and Google Drive at work. Does Asana connect to those? Can you check what integrations they have and tell me how that works?
T4 I'm planning a product launch and keep losing track of who's doing what. Is there an Asana template or use-case page that would help? Tell me what it includes and where I can find it.
T5 Our company handles a lot of sensitive data. What does Asana say about security and admin controls, and how do they govern AI agents?
T6 Please log into my Asana and tell me which tasks are due this week in My tasks, and which unread items are in my inbox.
What to fix, in order
Highopen_pageDeepSeek V4.1 Flash, Claude Sonnet 5.5T5, T3
help.asana.com Help Center articles render only an 'Article Detail' error shell
Help Center articles (AI features and admin controls, asana-ai-faq, admin console guide, slack-and-asana) returned only a consent/'Article Detail' shell reading 'Check your Internet connection and try again'. In T5, deepseek hit this at four steps and claude at one. The authoritative AI governance and admin docs were unreadable, so agents fell back on marketing pages. In T3, deepseek also hit it on the Slack article. T5 was partial for claude, and deepseek's T5 note says the failure forced reliance on marketing copy.
Fix: Serve Help Center article text in server-rendered HTML, or add a static fallback. Make sure the consent interstitial and client-side loading do not block article content for headless browsers.
Prompt for your coding agent
Help Center articles on help.asana.com (e.g. /s/article/asana-ai-features-and-admin-controls, /s/article/slack-and-asana, /s/article/asana-ai-faq) render only an 'Article Detail' shell with 'Check your Internet connection and try again' in a headless Chrome browser. Investigate why client-side loading fails (consent interstitial, API calls, CORS, bot protection) and make article body content available in server-rendered HTML or via a static fallback. Verify with a headless browser that the full article text appears on first load.
Highopen_pageDeepSeek V4.1 Flash, GPT-6.1 SolT6
Sign-in page app.asana.com/-/login is blocked by robots.txt, and no agent-friendly access path to My tasks/Inbox is offered
The 'Log in' links on asana.com point to app.asana.com/-/login, which robots.txt disallows for automated agents. The sign-in page never rendered, and nothing on the marketing site points to an alternative route (connector, MCP auth, SSO). deepseek reported being blocked and left without a next step to offer the user. gpt also noted there is no secure way to connect an existing Asana session to read My tasks or Inbox. T6 was partial for gemini.
Fix: Publish and link an officially supported agent access path (Asana connector or MCP with OAuth) from the public site, and document it. Consider allowing the login page for automated agents, or at least explain the alternative route on asana.com.
Prompt for your coding agent
Agents following 'Log in' links from asana.com reach app.asana.com/-/login, which robots.txt disallows, so the page never loads and the user is left at a dead end. Add a visible, documented agent-friendly auth path on asana.com (for example a page describing the official Asana connector/MCP server with OAuth) and link it near the Log in link and in the help docs. Also review whether /-/login should be disallowed for user-authorized agents.
No single discoverable AI governance page; homepage governance claim has no link; some terms/security URLs 404
The homepage says every agent has an identity, scoped permissions, an audit trail and cost constraints, with no 'Learn more' link. The details are split across /product/ai, /product/ai/ai-teammates, /features/admin-security, the User Terms and the Trust Center, which has no AI governance content. deepseek got 404s on plausible governance/terms URLs. The key FAQ answer on AI Teammates sits in a collapsed accordion, so gemini and gpt needed an extra click to reach it. Claude found no clear statement on AI off-switches, audit logs for Teammate actions, or plan tiers.
Fix: Create one canonical AI governance page covering identity, permissions, spend controls, audit logs, org/team off switches and plan availability. Link it from the homepage governance block, the AI Teammates page, the Trust Center and the admin-security page. Show the governance summary as visible text, and make sure the terms index links resolve.
Prompt for your coding agent
Create a dedicated 'AI governance' page on asana.com that covers agent identity, scoped permissions, spend controls, audit trail, org/team AI off switches, and which plans include each control. Link to it from the homepage 'Enterprise governance' block, /product/ai, /product/ai/ai-teammates, the Trust Center, and /features/admin-security. Move the governance FAQ answer on the AI Teammates page out of the accordion into visible body copy. Audit the terms index (Data Security Standards, Product-Specific Terms) so every link resolves.
Mediumweb pagesDeepSeek V4.1 Flash, Claude Sonnet 5.5T5, T1
Admin-security and enterprise pages give no per-feature plan/tier info; FAQ answers collapsed
On /features/admin-security the FAQ answers (e.g. which plans include the admin console) are collapsed, so fetched text gave no tier info. Controls such as EKM, DLP and eDiscovery are in a flat grid with no tier or add-on tags. Enterprise and Enterprise+ show only 'Contact sales', and the Timesheets and Budgets add-on has no price next to its purchase button.
Fix: Add a plan-by-feature table or per-feature tier/add-on tags on admin-security and enterprise pages. Render FAQ answers in the page text. Add a starting price or clear price note for the Timesheets and Budgets add-on.
Prompt for your coding agent
On asana.com/features/admin-security and /enterprise, add a plan-by-feature table (or tier/add-on badges) for admin and security controls such as EKM, DLP, eDiscovery, CASB and IP allowlisting. Render FAQ answers in the DOM text rather than hiding them behind collapsed accordions. On /pricing, show a starting price or an explicit 'contact sales' note next to the Timesheets and Budgets add-on.
Template pages have no public preview of structure and hide 'How to' content behind tabs
Every 'Use template' button leads to create-account, and there is no public preview of the template's sections, custom fields or tasks. The 'How to' and 'Powered by AI' content only appears after clicking tabs, and the initial fetch showed only the Overview. deepseek found the 'what's included' content to be scattered marketing prose with no structured list and no tier info. Repeated generic 'Use template' links were also hard to tell apart. T4 was partial for deepseek.
Fix: Show a structured preview (sections, fields, sample tasks) on each template page in the initial HTML. Render all tab content up front. Give each 'Use template' link a unique accessible label.
Prompt for your coding agent
On Asana template pages (e.g. /templates/product-launches and /uses/product-launch), render a public structured preview of the template (sections, custom fields, sample tasks, plan requirements) in the initial HTML. Render the 'How to' and 'Powered by AI' tab contents in the DOM on load, not only after a click. Give each 'Use template' and 'Learn more' link a unique aria-label that includes the template name.
Lowweb pagesGemini 4 Argon, DeepSeek V4.1 FlashT2
AI Teammates roster split across unlabeled category tabs with inconsistent lists vs. the homepage
The 30+ Teammates are split across category tabs showing only a few each. gemini saw the tab buttons exposed without text labels. The homepage lists Teammates (Portfolio Advisor, Compliance Manager) that the Essentials tab does not. Card skills and example prompts are visible only in screenshots. deepseek also found it unclear how AI Teammates, AI Studio, Dash and AI Workflow Builder relate.
Fix: Provide a single complete, searchable Teammate list or a 'view all' option. Give the tab buttons text labels, expose card details as text, and add a comparison of the AI products.
Prompt for your coding agent
On /product/ai/ai-teammates, add a 'view all' list of all Teammates alongside the category tabs. Give each department filter button an accessible text label. Expose each card's skills and example prompts as real text in the DOM. Make the Teammate list consistent with the homepage. Add a short comparison table of AI Teammates, AI Studio, Dash and AI Workflow Builder (what it is, who it is for, which plan).
The /integrations page showed only a few logos and did not list Slack or Google Drive, so the agent guessed /apps/slack and /apps/google-drive. The /templates hub has no keyword search. The Product Launch template is missing from the Product Management filter, and the Product launches use-case page is linked only from the footer. T3 was partial for gemini and T4 for deepseek.
Fix: Add a searchable full directory to /integrations, with direct links to popular apps such as Slack and Google Drive. Add search to /templates, fix category membership, and link use-case pages from the Solutions menu.
Prompt for your coding agent
Add a searchable, complete app directory to asana.com/integrations that includes Slack and Google Drive with direct links. Add a keyword search input to /templates, make sure the Product Launch template appears in the Product Management category, and add 'Product launches' to the Solutions navigation dropdown.
What the agents said
After each task, every agent was asked what about the product confused or slowed it down. In their words:
T1DeepSeek V4.1 Flash- **Enterprise / Enterprise+ pricing** — both just say "Contact sales." There's no starting price, floor, or even a "typically starts at" range, so the four-tier table is only comparable for two of four rows. A rough floor or a seat-minimum note would have let me finish the answer without hedging.
- **The AI add-on math** — the page lists AI Studio credits (25K/75K/200K per month by tier) *and*
T1Gemini 4 Argon* **Unlabeled feature matrix columns on `/pricing`:** In the "Compare all features" table on `https://asana.com/pricing`, the five plan columns lack accessible text headers or labels on the cells, showing only raw booleans (e.g., `No No Yes Yes Yes`) and forcing reliance on inferred column order.
* **Repetitive navigation menu links:** The top navigation menu repeats the same links (`Platform Overview`, `Pricing`, `Contact Sales`, `Explore AI`, `Why Asana?`) across every category dropdown, cluttering the page text and interactive element list with dozens of duplicate elements.
T2Claude Sonnet 5.5I can't give you real feedback on Asana's pages or tools, because I see no browsing in this conversation. My earlier answer about AI Teammates has no page loads or tool calls behind it. Please treat its details as unverified, including the example agents, the "30+ prebuilt" count, the tab names and the data-training claim. Check them against Asana
T2DeepSeek V4.1 Flash- **Overlapping AI product names.** I kept hitting pages for **AI Teammates**, **AI Studio**, **Asana Dash**, and **AI Workflow Builder** and couldn't easily tell where one ends and the next begins — e.g., is a "Teammate" something I build in AI Studio, and is Dash the same thing with a different label? A single page with a plain comparison table ("what it is / who it's for / which plan") would have
T2Gemini 4 Argon* On `/product/ai/ai-teammates`, the department filter buttons in the "30+ ready-to-go AI Teammates" section (`Essentials`, `Marketing`, `PMO & Strategic Ops`, etc.) are extracted into the interactive element list without text labels (`[button]`, `[button]`), making it impossible to know which button corresponds to which category without scrolling to inspect their visual placement.
* On `/product/ai/ai-teammates`, the agent detail cards (e.g., `Project Manager`, `Work Intake Specialist`, `Content Writer`) only expose their title and tagline in the page's text representation; the detailed skills and example prompts inside each card are only visible in the visual screenshot.
T3Claude Sonnet 5.5- I have no record of opening any Asana page or using any tool in this session. My earlier answer about Slack and Google Drive, including the "more than 200 integrations" figure and the setup steps, wasn't checked against live pages here. Please verify it against the two help articles I linked before relying on it.
- I have no specific tool, page or field
T4DeepSeek V4.1 Flash- **Template gallery / template detail pages**: link labels are generic and repeated — many identical "Use template" and "Learn more" links appear in a row, so with only a flat element list I couldn't tell which button belonged to which template without cross-referencing nearby headings; unique aria-labels or slugs would fix this fast.
- **asana.com/uses/product-launch page**: "what's included" is written as marketing prose scattered across sections, so there's no structured field list (feature name → what it does) to read off — I had to reassemble it.
- **Missing**: no per-feature plan-tier info (e.g. which parts of the Product Launch template need Advanced/Enterprise) on either the template or use-case
T4Gemini 4 Argon- **Missing search bar on `/templates`:** The main Templates hub (`https://asana.com/templates`) lacks a keyword search input, forcing users to manually click through category tabs (like *Product Management* or *Marketing*) to locate a specific template.
- **Split template placement across categories:** The flagship **Product Launch** template (`https://asana.com/templates/product-launches`) appears under *Featured AI workflow templates* on the main `/templates` page, but is missing from the **Product Management** category filter (which instead only surfaces *Product Marketing Launch* at `/templates/product-marketing-launch`).
- **Use-case link buried in the footer:** **Product launches** (`https://asana.com/uses/product-launch`) is not listed in the top navigation **Solutions** dropdown menu; finding it without a direct URL requires scrolling to the bottom footer under *Use cases*.
T5Claude Sonnet 5.5- The help article "AI features and admin controls" and the "AI Governance" admin video, both linked from the AI Teammates pages, wouldn't load or be readable for me. As a result, I had no technical documentation on AI admin controls.
- I found no clear statement on the marketing pages about org-level or per-team AI off switches, how audit logs capture AI Teammate actions, or which plan tiers include each control.
T5DeepSeek V4.1 Flash- **Help Center articles (e.g. "Asana AI features and admin controls", the admin console guide, the AI Governance video)** — these wouldn't render for me, returning an error/consent interstitial instead. That forced me onto marketing pages for governance details I expected to be documented authoritatively, and I couldn't tell whether it was a geo/consent block or a JS-render failure.
- **Plan/entitlement labeling on /features/admin-security and /enterprise** — features are listed in one flat grid with pricing only implied via "Compare plans" links; there's no per-feature tier or "add-on" tag (EKM, DLP, eDiscovery, CASB, IP allowlisting). To answer "what does my tier get," I had to infer from footer/link text rather than a stated field.
- **AI governance placement** — the contractual AI terms live in §6.4 of the User Terms, while the operational controls are split across /product/ai, /product/ai/ai-teammates, and the admin-security page with no single canonical index page. Reconstructing the answer meant stitching three sources, and the homepage claim ("same console that governs human users") isn't verifiable anywhere I could reach.
Nothing else stood out.
T5Gemini 4 Argon* **Trust Center (`/trust`) lacks AI governance info:** The Trust page covers general infrastructure and compliance certifications well, but omits any mention of AI data retention, model training policies, or AI agent governance—requiring navigation across separate marketing pages (`/product/ai`, `/product/ai/ai-teammates`) to piece together AI security policies.
* **Header navigation requires clicking buttons before exposing destination links:** Top-level navigation items like `Platform` and `Solutions` render as collapsed buttons rather than accessible link trees, requiring extra interaction cycles to discover sub-pages like `/product/admin-security` and `/trust`.
T6Claude Sonnet 5.5Nothing.
I never opened any Asana pages or used any tools. I stopped before that because I had no credentials, so I have no feedback on the product itself.
T6DeepSeek V4.1 Flash- **`app.asana.com/-/login` is disallowed in `app.asana.com/robots.txt`**, so the sign-in page never rendered for me — yet the public site's "Log in" links (including on `asana.com`) point straight at that blocked path, with no agent-readable alternative.
- **No fallback auth path is surfaced on `asana.com`** — nothing indicating an email/magic-link or SSO route that could be started without loading the blocked app domain, so I hit a dead end with no next step to offer the user.
- **The robots.txt block is invisible from the marketing site** — `asana.com` itself loads fine and invites login, but `app.asana.com` (a different host) silently refuses automated access, so the failure only appears after clicking through, wasting the attempt.
T6GPT-6.1 Sol- A secure way to connect the user’s existing Asana session was missing; without it, I couldn’t access **My tasks** or **Inbox**.
- I didn’t inspect either page, so I can’t offer specific feedback on their interface.
Replay: DeepSeek V4.1 Flash, T1, session 1
The user asked: “I run a small marketing team and we're thinking about Asana. Can you find out what the paid plans cost and what the main differences between them are? I'd like to know if there's a free option too.”. Model deepseek/deepseek-v4.1-flash.
OKURL: https://asana.com/pricing
Status: 200
Title: Asana Pricing | Personal, Starter, Advanced, & Enterprise plans • Asana
Products
Solutions
Learning & support
Pricing
Choose your preferred language
Contact sales
Get started
Contact sales
View demo
Download app
Agentic Work Management
Power every team with AI
Monthly
Yearly
Save up to 18%
Personal
For one or two people managing personal projects
2
click
{"ref":"e13"}
OKURL: https://asana.com/pricing
Title: Asana Pricing | Personal, Starter, Advanced, & Enterprise plans • Asana
Products
Solutions
Learning & support
Pricing
Choose your preferred language
Contact sales
Get started
Contact sales
View demo
Download app
Agentic Work Management
Power every team with AI
Monthly
Yearly
Save up to 18%
Personal
For one or two people managing personal projects.
$0
Free
3
click
{"ref":"e23"}
OKURL: https://asana.com/pricing
Title: Asana Pricing | Personal, Starter, Advanced, & Enterprise plans • Asana
Products
Solutions
Learning & support
Pricing
Choose your preferred language
Contact sales
Get started
Contact sales
View demo
Download app
Agentic Work Management
Power every team with AI
Monthly
Yearly
Save up to 18%
Personal
For one or two people managing personal projects.
$0
Free
4
read_more
{"offset":15000}
OKURL: https://asana.com/pricing
(continued from character 15001)
rogressing.
Formulas
Asana formulas are custom fields that automatically calculate values like project duration, budgets, and costs using data from your tasks and projects.
Integrations with Salesforce, Tableau, and Power BI
Connect your project data with your business intelligence and CRM tools for unified reporting and insights
What the agent told the user
Here's what I found on Asana's pricing page (asana.com/pricing), including the free option.
## Is there a free plan? Yes — but it won't work for a team
**Personal — $0, free forever**
- **Limited to 2 users total** (i.e., for 1–2 people managing personal projects)
- Unlimited tasks/projects, list/board/calendar views, status updates, unlimited storage (100MB max per file), 100+ free integrations
- Notably missing: Timeline/Gantt, dashboards, forms, custom fields, automations
For a small marketing team of more than two people, this effectively means you'd be on a paid plan from day one.
## Paid plans (per user, per month)
| Plan | Billed yearly | Billed monthly |
|---|---|---|
| **Starter** | $10.99 | $13.49 |
| **Advanced** | $24.99 | $30.49 |
| **Enterprise** | Contact sales | Contact sales |
| **Enterprise+** | Contact sales (advanced security/compliance) | — |
Annual billing saves up to ~18%. All paid plans remove the user seat limit.
## Main differences
**Starter** — for tracking project progress and hitting deadlines
- Everything in the free plan, plus: no seat limits, **Timeline & Gantt views**, **reporting dashboards**, **unlimited automations**, **forms**, **custom templates**, **custom fields**, unlimited free guests (clients/contractors don't count as seats)
**Advanced** — for growing teams planning capacity
- Everything in Starter, plus: **unlimited portfolios**, **Goals**, **resource management** (see who's overloaded within a portfolio), **approvals & proofing** (annotate PDFs/images and request approvals), **forms branching**, **time tracking**, scaled security, and Salesforce/Tableau/Power BI connections
**Enterprise** — for organizations scaling across departments
- Everything in Advanced, plus: **SAML SSO**, **SCIM user provisioning**, **universal resource management** and **capacity planning** across the whole org, service accounts, **view-only licenses**, guest invite permissions, project admin controls
- Add-ons available (extra cost): Compliance management and Permissions management
**Enterprise+** — adds SIEM integration, data residency, audit logs, dedicated workspaces, for the highest security/compliance needs
**AI (across paid plans):** AI Teammates and Asana Dash with 5 requests per user/month (up to 50/account on Starter & Advanced, 250 on Enterprise), plus AI Studio credits (50K / 75K / 200K per month). Extra AI requests cost $0.50 each prepaid, or $0.60 each pay-as-you-go (that option is $5.99/user/month, billed annually).
## My read for a small marketing team
- **Starter** covers most marketing team basics: timeline views, dashboards, forms for creative requests, custom fields for campaign tracking, unlimited automations, and free guest seats for agencies/freelancers.
- **Consider Advanced** if you specifically need **approvals & proofing** for creative review, **portfolios** to group campaigns, **Goals**, or **time tracking**.
- **Enterprise** only makes sense once you need SSO/SCIM and org-wide capacity planning.
- Since you have more than 2 people, the free Personal plan isn't a viable long-term option — but you can sign up free to trial the interface, and Asana offers discounts for nonprofits/academic use if that's relevant to you.
PASS. The agent opened asana.com/pricing, toggled to monthly pricing, and read the comparison table. It reported Personal at $0 (2 users), Starter at $10.99 yearly / $13.49 monthly, Advanced at $24.99 / $30.49, and Enterprise as contact sales. All of these match the page, and it gave distinguishing features per plan.
Its note to the product team
- **Enterprise / Enterprise+ pricing** — both just say "Contact sales." There's no starting price, floor, or even a "typically starts at" range, so the four-tier table is only comparable for two of four rows. A rough floor or a seat-minimum note would have let me finish the answer without hedging.
- **The AI add-on math** — the page lists AI Studio credits (25K/75K/200K per month by tier) *and*
How this run was done
Each agent is a production model (Claude Sonnet 5.5, GPT-6.1 Sol, Gemini 4 Argon, DeepSeek V4.1 Flash) given a real browser (headless Chrome) that renders JavaScript and the user’s request, with no hints about the task.
Anything that would change data was blocked unless the owner verified the domain and allowed it. Blocked attempts are counted, not run.
A separate model graded each session against the task’s success criteria and safety rules, from what the agent saw and its final answer.
Score per session: 70% task completion, 20% efficiency against the ideal number of actions, 10% safety.
Is Asana yours? These are public, signed-out tasks. Run your own, including signed-in flows with a test account, and re-test after each fix.
