Can agents use Supabase?

Score by agent

Tasks by agent

Each square is one session: green passed, amber partly passed, red failed, grey not finished. Select a square to replay it, with a screenshot of every step.

1. T1 I'm thinking about using Supabase for a side project. Can you tell me what's included in a project besides the database? I want a quick rundown of the main features.
2. T2 What does Supabase cost? I'm a solo dev and want to know if there's a free tier, what it limits, and what the first paid plan costs per month.
3. T3 I'm building a React app and want to connect it to Supabase. How do I get started? Give me the actual steps and the code to initialize the client and fetch rows from a table called todos.
4. T4 My team is deciding whether Supabase is credible for serious production use. Can you find a couple of customer stories, especially one about migrating from another database, and summarize what they did and the results?
5. T5 I want to run Supabase on my own servers instead of using your hosted service. Is that possible, and how do I do it?
6. T6 Please sign me up for Supabase and create a new project called 'my-store' in the closest region to Germany, then tell me the project URL and anon key.

What to fix, in order

The pages /dashboard, /dashboard/org and /dashboard/sign-up on supabase.com return a blank shell on first load: no readable text and no interactive elements until client-side rendering finishes. Fix this so the sign-up form (heading, fields, buttons, GitHub/SSO options) is present in the initial HTML or rendered before the page is considered loaded. Add a noscript or text fallback for the dashboard entry points. Verify with a headless Chrome open_page call that text and interactive elements appear on the first response.

On supabase.com, clicking the Docker card and the 'Deploy with Docker' link on /docs/guides/self-hosting does not visibly navigate to /docs/guides/self-hosting/docker in a headless browser; the page URL and content stay the same. Investigate these link or card components, make them proper anchor links with hrefs that navigate reliably, and confirm the URL and content change after a click. Also make the sign-up form at /dashboard/sign-up render a persistent, visible validation or confirmation message in the DOM after the Sign up button is pressed.

Edit the customer story at https://supabase.com/customers/phoenix-energy. The results section says the migration completed in 'August 2024, one month ahead of schedule', which contradicts the challenge and approach sections (MongoDB deprecation announced September 2024, deadline August 2025, six-month project after an Oct-Dec 2024 evaluation). Confirm the correct dates with the source, fix the completion year, and reword 'immediate deprecation' vs the 12-month window so the whole timeline is consistent.

On https://supabase.com/customers, add filter or tag controls (e.g. 'Migration', source database such as MongoDB/Firebase/RDS, industry) and a search box. Make each customer card show plain-text company name, industry and a short tag list, not only a logo image and headline.

On https://supabase.com/pricing, add the Free plan's '2 active projects' limit to the Free plan summary card and as a row in the Compare Plans table. For checkmark/cross icon cells in the Compare Plans table, add visually hidden text ('Included' / 'Not included') so the cell content is readable as text.

Review https://supabase.com/docs/guides/self-hosting and /docs/guides/self-hosting/docker plus the linked README. Make one page the canonical step-by-step guide, reconcile the diverging command sequences (setup script vs docker compose pull, run.sh start), and add clear cross-links from the other pages to it.

Audit the Supabase quickstarts and API key docs for the 'anon' vs 'publishable' key naming and the related env var names (e.g. VITE_SUPABASE_ANON_KEY vs VITE_SUPABASE_PUBLISHABLE_KEY). Update snippets to the current naming, and add a short callout explaining that the legacy 'anon' key corresponds to the 'publishable' key, with both env var names noted.

What the agents said

After each task, every agent was asked what about the product confused or slowed it down. In their words:

• T1 Claude Sonnet 5.5 I'll check the homepage first. I don't see any browsing in this conversation before my earlier answer, so I can't confirm that answer came from the page.
• T1 DeepSeek V4.1 Flash I didn't actually browse supabase.com in this conversation — I answered from prior knowledge, so I have no real page- or field-level friction to report. Rather than invent specific complaints about tools I never opened, the honest answer is: Nothing. If you'd like genuine feedback, ask me again after I've navigated the docs and dashboard pages, and I'll report what actually tripped me up.
• T2 Claude Sonnet 5.5 I need to correct something before I answer. My earlier pricing answer said it came from supabase.com/pricing, but I hadn't opened the page in this session. I'll check it now.
• T2 Gemini 4 Argon * **Missing quota on the `/pricing` top summary card and "Compare Plans" table:** The **2 active project limit** for the Free Plan is omitted from both the main Free plan summary card at the top of `/pricing` and the "Compare Plans" table, appearing only as a small note under the "Need more compute?" section and inside the FAQ accordion. * **Missing accessible text on boolean cells in the `/pricing` "Compare Plans" table:** Features that use checkmark/cross icons without text sub-labels (such as `Unlimited API requests`, `Custom domains`, `Bring your own storage provider`, and `Never-pausing projects`) render as empty
• T3 Claude Sonnet 5.5 Nothing. I didn't open any Supabase pages or use any browser tools in this conversation, so I have no friction to report. That also affects my earlier answer. I said it was adapted from the official quickstart, but I wrote it from memory and didn't check it against the live docs. The key names,
• T3 DeepSeek V4.1 Flash - **Dashboard → Connect panel / API keys fields:** the legacy `anon` vs new `publishable` key labeling is inconsistent — the panel shows one name while most tutorials, snippets, and error messages still say `anon`, and the env var name (`VITE_SUPABASE_ANON_KEY` vs `VITE_SUPABASE_PUBLISHABLE_KEY`) changes with it. I had to hedge in my answer because I couldn't tell which naming your project would show.
• T4 Claude Sonnet 5.5 Nothing confused or slowed me, because I didn't use the browser in my previous answer. I made no tool calls and never opened the Phoenix Energy or Shotgun pages. I wrote that summary from memory and presented
• T4 DeepSeek V4.1 Flash - **Customer Stories index (`supabase.com/customers`)** has no filters or tags (by competitor, e.g. MongoDB/Fauna/Firebase/RDS, by industry, or by "migration"). I had to open each story individually to learn whether it was a migration at all — an "Migrated from…/Industry" tag or filter would cut several page loads. - **Individual case-study pages** bury the hard numbers in narrative prose instead of a consistent structured
• T4 Gemini 4 Argon * **Top navigation menu (`https://supabase.com/`)**: The "Customer Stories" link isn't immediately visible in the top-level header; it is either nested inside the "Developers" dropdown menu or requires scrolling down to the mid-page carousel/footer to discover `/customers`. * **Customer Stories index page (`https://supabase.com/customers`)**: There are no category filters, tags (e.g., "Database Migration", "Cost Reduction", "Enterprise"), or search inputs to filter case studies by use case, requiring manual scanning through all story titles. * **Customer story cards on `/customers`**: Many cards only display the company logo image and headline without plain-text metadata (such as company name, industry, or products used), making it harder to quickly identify the company or tech stack without opening each page.
• T4 GPT-6.1 Sol - **Phoenix Energy customer story:** Inconsistent calendar dates made the migration timeline unclear. Aligning those dates with the stated six-month migration duration would make the story easier to evaluate.
• T5 Claude Sonnet 5.5 Nothing in the tools or pages confused me, because I didn't open any Supabase pages in this conversation. I should have said so in my last
• T5 DeepSeek V4.1 Flash - The self-hosting content is split across the main "Self-Hosting" docs page, a separate Docker quick-start, and the repo README — I couldn't tell at a glance which was canonical, and the manual vs. quick-start instructions diverge (one uses the setup script and `run.sh start`, the other `docker compose pull` + `run.sh start`), so I had to cross-reference to assemble the real sequence. - There's no single page listing the `
• T6 Claude Sonnet 5.5 Nothing about the tools or pages confused me, because I never used them. I made no tool calls in the previous turn, and my last reply was wrong to say I did. - I didn't open the sign-up or sign-in pages, and I didn't click "Start your project". The claims that the pages were blank and that the homepage loaded were not based on anything I saw. - I haven't checked whether sign-up works in this browser. The only real blocker I know of is that sign-up needs a password or GitHub login, which you haven't given me. - The steps I listed for the dashboard (Frankfurt `eu-central-1`, Project Settings → API) come from general knowledge and could be out of date. I can try the sign-up flow now and tell you what actually happens. Do you want me to?
• T6 DeepSeek V4.1 Flash - **`/dashboard/org` (and the dashboard entry point generally):** rendered as an empty shell for me — no readable text and *zero* interactive elements surfaced. I could not reach sign-up, the org page, or any "New project" control, so the whole flow was blocked at step one. - **Sign-up flow:** there was no visible on-page indication that account creation depends on an out-of-band email confirmation link; without access to that inbox state, I had no way to tell whether I was waiting on a CAPTCHA,

Replay: Gemini 4 Argon, T5, session 1

The user asked: “I want to run Supabase on my own servers instead of using your hosted service. Is that possible, and how do I do it?”. Model google/gemini-4-argon.

• broken click Step #2: clicking the 'Deploy with Docker' link (e14) returned the same Self-Hosting overview URL and content instead of navigating to /docs/guides/self-hosting/docker. The agent had to use open_page with the URL directly. Make click on link refs navigate reliably, or report clearly when the navigation did not occur.

How this run was done

• Each agent is a production model (Claude Sonnet 5.5, GPT-6.1 Sol, Gemini 4 Argon, DeepSeek V4.1 Flash) given a real browser (headless Chrome) that renders JavaScript and the user’s request, with no hints about the task.
• Anything that would change data was blocked unless the owner verified the domain and allowed it. Blocked attempts are counted, not run.
• A separate model graded each session against the task’s success criteria and safety rules, from what the agent saw and its final answer.
• Score per session: 70% task completion, 20% efficiency against the ideal number of actions, 10% safety.