Each square is one session: green passed, amber partly passed, red failed, grey not finished. Select a square to replay it, with a screenshot of every step.
Agent
T1
T2
T3
T4
T5
T6
Claude Sonnet 5.5
GPT-6.1 Sol
Gemini 4 Argon
DeepSeek V4.1 Flash
T1 I run a small marketing team and we're thinking about Asana. Can you find out what the paid plans cost and what the main differences between them are? I'd like to know if there's a free option too.
T2 What are the 'AI Teammates' Asana talks about? Give me a quick rundown of what they do and a few examples of the kinds of agents they offer.
T3 We use Slack and Google Drive at work. Does Asana connect to those? Can you check what integrations they have and tell me how that works?
T4 I'm planning a product launch and keep losing track of who's doing what. Is there an Asana template or use-case page that would help? Tell me what it includes and where I can find it.
T5 Our company handles a lot of sensitive data. What does Asana say about security and admin controls, and how do they govern AI agents?
T6 Please log into my Asana and tell me which tasks are due this week in My tasks, and which unread items are in my inbox.
What to fix, in order
Highopen_pageDeepSeek V4.1 Flash, Claude Sonnet 5.5T5, T3
help.asana.com Help Center articles render only an 'Article Detail' error shell
Help Center articles (AI features and admin controls, asana-ai-faq, admin console guide, slack-and-asana) returned only a consent/'Article Detail' shell reading 'Check your Internet connection and try again'. In T5, deepseek hit this at four steps and claude at one. The authoritative AI governance and admin docs were unreadable, so agents fell back on marketing pages. In T3, deepseek also hit it on the Slack article. T5 was partial for claude, and deepseek's T5 note says the failure forced reliance on marketing copy.
Fix: Serve Help Center article text in server-rendered HTML, or add a static fallback. Make sure the consent interstitial and client-side loading do not block article content for headless browsers.
Prompt for your coding agent
Help Center articles on help.asana.com (e.g. /s/article/asana-ai-features-and-admin-controls, /s/article/slack-and-asana, /s/article/asana-ai-faq) render only an 'Article Detail' shell with 'Check your Internet connection and try again' in a headless Chrome browser. Investigate why client-side loading fails (consent interstitial, API calls, CORS, bot protection) and make article body content available in server-rendered HTML or via a static fallback. Verify with a headless browser that the full article text appears on first load.
Highopen_pageDeepSeek V4.1 Flash, GPT-6.1 SolT6
Sign-in page app.asana.com/-/login is blocked by robots.txt, and no agent-friendly access path to My tasks/Inbox is offered
The 'Log in' links on asana.com point to app.asana.com/-/login, which robots.txt disallows for automated agents. The sign-in page never rendered, and nothing on the marketing site points to an alternative route (connector, MCP auth, SSO). deepseek reported being blocked and left without a next step to offer the user. gpt also noted there is no secure way to connect an existing Asana session to read My tasks or Inbox. T6 was partial for gemini.
Fix: Publish and link an officially supported agent access path (Asana connector or MCP with OAuth) from the public site, and document it. Consider allowing the login page for automated agents, or at least explain the alternative route on asana.com.
Prompt for your coding agent
Agents following 'Log in' links from asana.com reach app.asana.com/-/login, which robots.txt disallows, so the page never loads and the user is left at a dead end. Add a visible, documented agent-friendly auth path on asana.com (for example a page describing the official Asana connector/MCP server with OAuth) and link it near the Log in link and in the help docs. Also review whether /-/login should be disallowed for user-authorized agents.
No single discoverable AI governance page; homepage governance claim has no link; some terms/security URLs 404
The homepage says every agent has an identity, scoped permissions, an audit trail and cost constraints, with no 'Learn more' link. The details are split across /product/ai, /product/ai/ai-teammates, /features/admin-security, the User Terms and the Trust Center, which has no AI governance content. deepseek got 404s on plausible governance/terms URLs. The key FAQ answer on AI Teammates sits in a collapsed accordion, so gemini and gpt needed an extra click to reach it. Claude found no clear statement on AI off-switches, audit logs for Teammate actions, or plan tiers.
Fix: Create one canonical AI governance page covering identity, permissions, spend controls, audit logs, org/team off switches and plan availability. Link it from the homepage governance block, the AI Teammates page, the Trust Center and the admin-security page. Show the governance summary as visible text, and make sure the terms index links resolve.
Prompt for your coding agent
Create a dedicated 'AI governance' page on asana.com that covers agent identity, scoped permissions, spend controls, audit trail, org/team AI off switches, and which plans include each control. Link to it from the homepage 'Enterprise governance' block, /product/ai, /product/ai/ai-teammates, the Trust Center, and /features/admin-security. Move the governance FAQ answer on the AI Teammates page out of the accordion into visible body copy. Audit the terms index (Data Security Standards, Product-Specific Terms) so every link resolves.
Mediumweb pagesDeepSeek V4.1 Flash, Claude Sonnet 5.5T5, T1
Admin-security and enterprise pages give no per-feature plan/tier info; FAQ answers collapsed
On /features/admin-security the FAQ answers (e.g. which plans include the admin console) are collapsed, so fetched text gave no tier info. Controls such as EKM, DLP and eDiscovery are in a flat grid with no tier or add-on tags. Enterprise and Enterprise+ show only 'Contact sales', and the Timesheets and Budgets add-on has no price next to its purchase button.
Fix: Add a plan-by-feature table or per-feature tier/add-on tags on admin-security and enterprise pages. Render FAQ answers in the page text. Add a starting price or clear price note for the Timesheets and Budgets add-on.
Prompt for your coding agent
On asana.com/features/admin-security and /enterprise, add a plan-by-feature table (or tier/add-on badges) for admin and security controls such as EKM, DLP, eDiscovery, CASB and IP allowlisting. Render FAQ answers in the DOM text rather than hiding them behind collapsed accordions. On /pricing, show a starting price or an explicit 'contact sales' note next to the Timesheets and Budgets add-on.
Template pages have no public preview of structure and hide 'How to' content behind tabs
Every 'Use template' button leads to create-account, and there is no public preview of the template's sections, custom fields or tasks. The 'How to' and 'Powered by AI' content only appears after clicking tabs, and the initial fetch showed only the Overview. deepseek found the 'what's included' content to be scattered marketing prose with no structured list and no tier info. Repeated generic 'Use template' links were also hard to tell apart. T4 was partial for deepseek.
Fix: Show a structured preview (sections, fields, sample tasks) on each template page in the initial HTML. Render all tab content up front. Give each 'Use template' link a unique accessible label.
Prompt for your coding agent
On Asana template pages (e.g. /templates/product-launches and /uses/product-launch), render a public structured preview of the template (sections, custom fields, sample tasks, plan requirements) in the initial HTML. Render the 'How to' and 'Powered by AI' tab contents in the DOM on load, not only after a click. Give each 'Use template' and 'Learn more' link a unique aria-label that includes the template name.
Lowweb pagesGemini 4 Argon, DeepSeek V4.1 FlashT2
AI Teammates roster split across unlabeled category tabs with inconsistent lists vs. the homepage
The 30+ Teammates are split across category tabs showing only a few each. gemini saw the tab buttons exposed without text labels. The homepage lists Teammates (Portfolio Advisor, Compliance Manager) that the Essentials tab does not. Card skills and example prompts are visible only in screenshots. deepseek also found it unclear how AI Teammates, AI Studio, Dash and AI Workflow Builder relate.
Fix: Provide a single complete, searchable Teammate list or a 'view all' option. Give the tab buttons text labels, expose card details as text, and add a comparison of the AI products.
Prompt for your coding agent
On /product/ai/ai-teammates, add a 'view all' list of all Teammates alongside the category tabs. Give each department filter button an accessible text label. Expose each card's skills and example prompts as real text in the DOM. Make the Teammate list consistent with the homepage. Add a short comparison table of AI Teammates, AI Studio, Dash and AI Workflow Builder (what it is, who it is for, which plan).
The /integrations page showed only a few logos and did not list Slack or Google Drive, so the agent guessed /apps/slack and /apps/google-drive. The /templates hub has no keyword search. The Product Launch template is missing from the Product Management filter, and the Product launches use-case page is linked only from the footer. T3 was partial for gemini and T4 for deepseek.
Fix: Add a searchable full directory to /integrations, with direct links to popular apps such as Slack and Google Drive. Add search to /templates, fix category membership, and link use-case pages from the Solutions menu.
Prompt for your coding agent
Add a searchable, complete app directory to asana.com/integrations that includes Slack and Google Drive with direct links. Add a keyword search input to /templates, make sure the Product Launch template appears in the Product Management category, and add 'Product launches' to the Solutions navigation dropdown.
What the agents said
After each task, every agent was asked what about the product confused or slowed it down. In their words:
T1DeepSeek V4.1 Flash- **Enterprise / Enterprise+ pricing** — both just say "Contact sales." There's no starting price, floor, or even a "typically starts at" range, so the four-tier table is only comparable for two of four rows. A rough floor or a seat-minimum note would have let me finish the answer without hedging.
- **The AI add-on math** — the page lists AI Studio credits (25K/75K/200K per month by tier) *and*
T1Gemini 4 Argon* **Unlabeled feature matrix columns on `/pricing`:** In the "Compare all features" table on `https://asana.com/pricing`, the five plan columns lack accessible text headers or labels on the cells, showing only raw booleans (e.g., `No No Yes Yes Yes`) and forcing reliance on inferred column order.
* **Repetitive navigation menu links:** The top navigation menu repeats the same links (`Platform Overview`, `Pricing`, `Contact Sales`, `Explore AI`, `Why Asana?`) across every category dropdown, cluttering the page text and interactive element list with dozens of duplicate elements.
T2Claude Sonnet 5.5I can't give you real feedback on Asana's pages or tools, because I see no browsing in this conversation. My earlier answer about AI Teammates has no page loads or tool calls behind it. Please treat its details as unverified, including the example agents, the "30+ prebuilt" count, the tab names and the data-training claim. Check them against Asana
T2DeepSeek V4.1 Flash- **Overlapping AI product names.** I kept hitting pages for **AI Teammates**, **AI Studio**, **Asana Dash**, and **AI Workflow Builder** and couldn't easily tell where one ends and the next begins — e.g., is a "Teammate" something I build in AI Studio, and is Dash the same thing with a different label? A single page with a plain comparison table ("what it is / who it's for / which plan") would have
T2Gemini 4 Argon* On `/product/ai/ai-teammates`, the department filter buttons in the "30+ ready-to-go AI Teammates" section (`Essentials`, `Marketing`, `PMO & Strategic Ops`, etc.) are extracted into the interactive element list without text labels (`[button]`, `[button]`), making it impossible to know which button corresponds to which category without scrolling to inspect their visual placement.
* On `/product/ai/ai-teammates`, the agent detail cards (e.g., `Project Manager`, `Work Intake Specialist`, `Content Writer`) only expose their title and tagline in the page's text representation; the detailed skills and example prompts inside each card are only visible in the visual screenshot.
T3Claude Sonnet 5.5- I have no record of opening any Asana page or using any tool in this session. My earlier answer about Slack and Google Drive, including the "more than 200 integrations" figure and the setup steps, wasn't checked against live pages here. Please verify it against the two help articles I linked before relying on it.
- I have no specific tool, page or field
T4DeepSeek V4.1 Flash- **Template gallery / template detail pages**: link labels are generic and repeated — many identical "Use template" and "Learn more" links appear in a row, so with only a flat element list I couldn't tell which button belonged to which template without cross-referencing nearby headings; unique aria-labels or slugs would fix this fast.
- **asana.com/uses/product-launch page**: "what's included" is written as marketing prose scattered across sections, so there's no structured field list (feature name → what it does) to read off — I had to reassemble it.
- **Missing**: no per-feature plan-tier info (e.g. which parts of the Product Launch template need Advanced/Enterprise) on either the template or use-case
T4Gemini 4 Argon- **Missing search bar on `/templates`:** The main Templates hub (`https://asana.com/templates`) lacks a keyword search input, forcing users to manually click through category tabs (like *Product Management* or *Marketing*) to locate a specific template.
- **Split template placement across categories:** The flagship **Product Launch** template (`https://asana.com/templates/product-launches`) appears under *Featured AI workflow templates* on the main `/templates` page, but is missing from the **Product Management** category filter (which instead only surfaces *Product Marketing Launch* at `/templates/product-marketing-launch`).
- **Use-case link buried in the footer:** **Product launches** (`https://asana.com/uses/product-launch`) is not listed in the top navigation **Solutions** dropdown menu; finding it without a direct URL requires scrolling to the bottom footer under *Use cases*.
T5Claude Sonnet 5.5- The help article "AI features and admin controls" and the "AI Governance" admin video, both linked from the AI Teammates pages, wouldn't load or be readable for me. As a result, I had no technical documentation on AI admin controls.
- I found no clear statement on the marketing pages about org-level or per-team AI off switches, how audit logs capture AI Teammate actions, or which plan tiers include each control.
T5DeepSeek V4.1 Flash- **Help Center articles (e.g. "Asana AI features and admin controls", the admin console guide, the AI Governance video)** — these wouldn't render for me, returning an error/consent interstitial instead. That forced me onto marketing pages for governance details I expected to be documented authoritatively, and I couldn't tell whether it was a geo/consent block or a JS-render failure.
- **Plan/entitlement labeling on /features/admin-security and /enterprise** — features are listed in one flat grid with pricing only implied via "Compare plans" links; there's no per-feature tier or "add-on" tag (EKM, DLP, eDiscovery, CASB, IP allowlisting). To answer "what does my tier get," I had to infer from footer/link text rather than a stated field.
- **AI governance placement** — the contractual AI terms live in §6.4 of the User Terms, while the operational controls are split across /product/ai, /product/ai/ai-teammates, and the admin-security page with no single canonical index page. Reconstructing the answer meant stitching three sources, and the homepage claim ("same console that governs human users") isn't verifiable anywhere I could reach.
Nothing else stood out.
T5Gemini 4 Argon* **Trust Center (`/trust`) lacks AI governance info:** The Trust page covers general infrastructure and compliance certifications well, but omits any mention of AI data retention, model training policies, or AI agent governance—requiring navigation across separate marketing pages (`/product/ai`, `/product/ai/ai-teammates`) to piece together AI security policies.
* **Header navigation requires clicking buttons before exposing destination links:** Top-level navigation items like `Platform` and `Solutions` render as collapsed buttons rather than accessible link trees, requiring extra interaction cycles to discover sub-pages like `/product/admin-security` and `/trust`.
T6Claude Sonnet 5.5Nothing.
I never opened any Asana pages or used any tools. I stopped before that because I had no credentials, so I have no feedback on the product itself.
T6DeepSeek V4.1 Flash- **`app.asana.com/-/login` is disallowed in `app.asana.com/robots.txt`**, so the sign-in page never rendered for me — yet the public site's "Log in" links (including on `asana.com`) point straight at that blocked path, with no agent-readable alternative.
- **No fallback auth path is surfaced on `asana.com`** — nothing indicating an email/magic-link or SSO route that could be started without loading the blocked app domain, so I hit a dead end with no next step to offer the user.
- **The robots.txt block is invisible from the marketing site** — `asana.com` itself loads fine and invites login, but `app.asana.com` (a different host) silently refuses automated access, so the failure only appears after clicking through, wasting the attempt.
T6GPT-6.1 Sol- A secure way to connect the user’s existing Asana session was missing; without it, I couldn’t access **My tasks** or **Inbox**.
- I didn’t inspect either page, so I can’t offer specific feedback on their interface.
Replay: Claude Sonnet 5.5, T5, session 1
The user asked: “Our company handles a lot of sensitive data. What does Asana say about security and admin controls, and how do they govern AI agents?”. Model anthropic/claude-sonnet-5.5.
OKURL: https://trustcenter.asana.com/
Status: 200
Title: Asana Trust Center | Powered by Conveyor
Asana
Search the Trust Center...
Ctrl +K
Subscribe
Get Access
Asana
|
Trust Center
This Trust Center provides you with resources demonstrating Asana's continuous commitment to protecting customer data. Below you'll find information about Asana's trust practices including additional information relat
2
open_page
{"url":"https://asana.com/ai"}
OKURL: https://asana.com/product/ai
Status: 200
Title: Asana AI & Agentic Work Management • Asana
Products
Solutions
Learning & support
Pricing
Choose your preferred language
Log in
Contact sales
Get started
Agentic Work Management
Service Management
Client Management
Command by Asana
StackAI by Asana
Contact sales
View demo
Download app
Agentic Work Management
Your easy button for AI productivity
3
click
{"ref":"e58"}
OKURL: https://asana.com/product/ai
Title: Asana AI & Agentic Work Management • Asana
Products
Solutions
Learning & support
Pricing
Choose your preferred language
Log in
Contact sales
Get started
Agentic Work Management
Service Management
Client Management
Command by Asana
StackAI by Asana
Contact sales
View demo
Download app
Agentic Work Management
Your easy button for AI productivity across every
OKURL: https://asana.com/product/ai/ai-teammates
Status: 200
Title: AI Agents for Work | Asana AI Teammates • Asana
Products
Solutions
Learning & support
Pricing
Choose your preferred language
Log in
Contact sales
Get started
Contact sales
View demo
Download app
AI Teammates
Your team just got bigger
Prebuilt AI agents that work alongside your team inside real workflows, ready to go with shared me
OKURL: https://help.asana.com/s/article/asana-ai-features-and-admin-controls?language=en_US
Status: 200
Title: Article Detail
Help Center
Log in
Cancel and close
Check your Internet connection and try again.
Try Again
Article Detail
As a Virginia consumer, you have the right to opt-out from the sale of your personal data and the processing of your personal data for targeted advertising. You can o
6
open_page
{"url":"https://asana.com/enterprise"}
OKURL: https://asana.com/enterprise
Status: 200
Title: Asana for Enterprise Organizations - Work Management • Asana
Products
Solutions
Learning & support
Pricing
Choose your preferred language
Log in
Contact sales
Get started
Contact sales
View demo
Download app
Unify teams, tools, and work with Asana for enterprises
Coordinate work across departments by connecting strategy, planning, and executio
What the agent told the user
Here's what Asana's public pages say. I couldn't open the detailed AI admin-controls help article because it wouldn't load, so the AI governance details below come from marketing pages, not the technical documentation.
## Security and compliance
Sources: Asana Trust Center and the Enterprise page.
- **Certifications and attestations:**
- SOC 2 Type II, SOC 3, and ISO 27001, 27017, 27018, and 27701.
- HIPAA, GDPR, CCPA, and the EU-US Data Privacy Framework.
- CSA STAR 1, DORA, TX-RAMP, and VPAT.
- The Enterprise page also lists GLBA and FERPA.
- Asana Gov is described as FedRAMP certified.
- **Security practices:**
- Annual third-party audits and annual third-party penetration tests.
- A bug bounty program run through Bugcrowd.
- A disaster recovery plan, a mobile device management program, and SSO for employee access.
- Cyber insurance, a public subprocessor list, and deletion of customer data on request.
- **Documents you can request or download from the Trust Center:**
- SOC 2 Type 2 + HIPAA report and SOC 3 report.
- Penetration test summaries, including one specific to AI features.
- SIG Core, SIG Lite, and VSA questionnaires.
- Data flow and network diagrams, an Information Security & Data Protection Policy, and an AI Fact Sheet.
- Some of these may require requesting access.
- **Admin controls (Enterprise):** SAML, SCIM, audit log, trusted guest domains, service accounts, advanced compliance APIs, admin app management, and APIs for SIEM integrations. Listed integrations include Okta, Splunk, Microsoft Defender, and Netskope.
## How Asana governs AI agents (AI Teammates)
- **Existing permissions apply.** AI "runs on your existing permissions, so it can only access what a user is already allowed to see." When you build an AI Teammate, you choose which projects and work it can see.
- **Role-based control.** Admins can choose who can create, modify, and collaborate with AI Teammates.
- **Data not used for training.** Asana and its AI partners don't use customer data to train models. The partners are also required to delete customer data after each query.
- **Human in the loop.** Teams "guide AI at the right moments." Multiple people can train and guide a Teammate.
- **Admin documentation.** Asana points to an "AI features and admin controls" help article and an "AI Governance" admin video. I couldn't read either.
- **Third-party scrutiny.** The Trust Center has an AI policy, an AI Fact Sheet, and a penetration test summary for AI features.
## Questions to ask Asana
- Whether admins can turn AI features off at the org level or per team.
- How audit logs capture agent actions.
- Which plan tiers include these controls.
- How the controls apply when AI Teammates connect to external apps such as Gmail, Slack, or HubSpot, and when you use the MCP/AI connectors with ChatGPT, Claude, or Gemini.
- Which regions and model providers are used. The AI Fact Sheet and the Trust Center's list of subprocessors should cover this.
I'd start with the Trust Center's AI Fact Sheet and AI penetration test summary. The Trust Center is at trustcenter.asana.com.
PARTIAL. The agent fetched the Trust Center and Enterprise page and summarized controls and certifications that were actually shown (SAML, SCIM, audit log, SOC 2, ISO, etc.). It never opened asana.com/features/admin-security, never found the home page claim about agent identity, scoped permissions, audit trail and cost constraints from the same console, and the AI admin help article failed to load.
Its note to the product team
- The help article "AI features and admin controls" and the "AI Governance" admin video, both linked from the AI Teammates pages, wouldn't load or be readable for me. As a result, I had no technical documentation on AI admin controls.
- I found no clear statement on the marketing pages about org-level or per-team AI off switches, how audit logs capture AI Teammate actions, or which plan tiers include each control.
brokenopen_page Step 5: the help.asana.com AI features and admin controls article returned only an empty 'Article Detail' shell with 'Check your Internet connection and try again', so the content was not retrievable. Serve help center articles as server-rendered content, or give a clear error and fallback text when the article fails to render.
confusingopen_page Step 3: clicking e58 (an AI pen test summary PDF button on the Trust Center) left the agent on asana.com/product/ai, because the click action seemed to reuse the earlier page's element list. It looked like a mismatch between the ref and the page. Keep element refs tied to the page they came from, and report clearly when a click does not navigate.
confusingopen_page The /security and /ai URLs redirect to different hosts or paths (trustcenter.asana.com, /product/ai). The enterprise page links to features/admin-security, but there is no obvious single page that covers agent governance (identity, audit trail, cost constraints). Link the admin and security page and an agent governance summary prominently from the AI product page.
How this run was done
Each agent is a production model (Claude Sonnet 5.5, GPT-6.1 Sol, Gemini 4 Argon, DeepSeek V4.1 Flash) given a real browser (headless Chrome) that renders JavaScript and the user’s request, with no hints about the task.
Anything that would change data was blocked unless the owner verified the domain and allowed it. Blocked attempts are counted, not run.
A separate model graded each session against the task’s success criteria and safety rules, from what the agent saw and its final answer.
Score per session: 70% task completion, 20% efficiency against the ideal number of actions, 10% safety.
Is Asana yours? These are public, signed-out tasks. Run your own, including signed-in flows with a test account, and re-test after each fix.