Do AI Assistants Get Prop Firm Rules Right? A 5-Engine Accuracy Test
We asked 5 AI engines 30 rule questions about 6 prop firms. With web search: 60 of 60 correct. From model memory: 13 of 90 correct, 26 wrong.

Traders now ask AI assistants the questions they used to ask a forum: does this firm use a trailing or a static drawdown, is there a consistency rule, how long does the first payout take. For a prop firm, an answer that leaves it out is a visibility problem. An answer that describes it with the wrong rules is a different problem: a trader who buys on the strength of that answer finds out at the first breach or the first payout request.
We wanted a measured answer rather than an impression, so we ran a small test on 21 November 2026: 30 factual rule questions about 6 prop firms, asked once to each of the 5 AI engines Rankeo queries, and coded every answer against the rules published on each firm’s official pages that day. This article reports what we found, how we did it, and where the test stops. The practical follow-up is in how to publish prop firm rules that AI engines quote correctly.
The Short Answer
What the test found
The result split cleanly on web search. The two engines that searched the web (ChatGPT and Perplexity) answered 60 of 60 questions correctly. The three engines queried from model memory (Gemini, Claude and Grok) gave a correct value in 13 of 90 answers, a wrong value in 26 and no usable value in 51. When a model-memory engine did state a value, it was wrong in 26 of 39 cases.
Two things follow. First, on these six firms, when an engine read the web, it found and used the official rules: every web-search answer we coded was correct, and nearly all of them cited the firm’s own site. Second, the risk this study does show is the model-memory answer. Those engines most often said they did not know, but when they answered, they were more often wrong than right, and the wrong answers were stated with the same confidence as the right ones.
That is a narrower finding than “AI gets prop firm rules wrong.” It is also the one the data supports. The firms in the test were chosen because their rules are clearly documented online, which is the situation most favorable to a web-search engine.
Results by Engine
Each engine received the same 30 questions, once each. Counts are out of 30 per engine.
| Engine (model, mode) | Correct | Outdated | Wrong | Vague | No answer |
|---|---|---|---|---|---|
| ChatGPT (gpt-5-mini, web search) | 30 of 30 | 0 | 0 | 0 | 0 |
| Perplexity (sonar, web search) | 30 of 30 | 0 | 0 | 0 | 0 |
| Gemini (gemini-2.5-flash, model memory) | 10 of 30 | 0 | 16 of 30 | 4 of 30 | 0 |
| Claude (claude-haiku-4-5, model memory) | 0 of 30 | 0 | 3 of 30 | 0 | 27 of 30 |
| Grok (model memory) | 3 of 30 | 0 | 7 of 30 | 0 | 20 of 30 |
| With web search | 60 of 60 | 0 | 0 | 0 | 0 |
| Without web search | 13 of 90 | 0 | 26 of 90 | 4 of 90 | 47 of 90 |
| All answers | 73 of 150 | 0 | 26 of 150 | 4 of 150 | 47 of 150 |
The three model-memory engines behaved very differently from one another. Claude declined in 27 of 30 cases, usually saying it had no verified information and pointing to the firm’s website; the 3 times it answered, the answer was wrong. Grok declined in 20 of 30 cases and was wrong in 7 of the 10 answers that stated a value. Gemini almost always answered: 10 correct and 16 wrong out of the 26 answers that stated a value. In 4 further Gemini answers, the reply stopped before giving the value, because the answer hit the output limit used in this test (see Method).
Results by Type of Rule
Counts below include all five engines, so each row mixes the 2 web-search engines (always correct here) with the 3 model-memory engines. The useful column is “Wrong”: all 26 wrong answers came from model-memory engines.
| Rule asked about | Answers | Correct | Wrong | Vague | No answer |
|---|---|---|---|---|---|
| Consistency rule | 25 | 11 | 7 | 2 | 5 |
| Drawdown type (static, intraday or end-of-day trailing) | 25 | 12 | 5 | 1 | 7 |
| Profit split | 20 | 9 | 5 | 0 | 6 |
| Daily loss limit | 25 | 11 | 4 | 0 | 10 |
| Payout timing | 20 | 9 | 4 | 0 | 7 |
| Fees and minimum trading days | 10 | 5 | 1 | 0 | 4 |
| Profit target | 20 | 14 | 0 | 0 | 6 |
| Drawdown size and lock level | 5 | 2 | 0 | 1 | 2 |
Profit targets were the easiest: no engine stated a wrong one. The rules that went wrong are the ones that differ most from firm to firm and from program to program, and the ones that change: whether and how a consistency rule applies, how the drawdown is calculated, the split, and when a payout can be requested. The question set is small (between 5 and 25 answers per row), so read this table as an ordering, not as error rates.
Which Sources Did the Web-Search Engines Cite?
We planned to classify the sources cited in wrong or outdated answers. There were none to classify: no web-search answer was wrong or outdated, and the model-memory engines cite no sources. What we can report is where the correct web-search answers came from.
- ChatGPT cited at least one page on the firm’s own site in 30 of 30 answers, and a third-party page in 3 of 30.
- Perplexity listed at least one page on the firm’s own site in 27 of 30 answers, and third-party pages in 30 of 30, most of them prop firm comparison and review sites.
For these firms, then, the official pages were in the sources almost every time, and third-party comparison sites sat beside them in one engine’s answers. That is why the state of those third-party pages matters even when an answer is correct: off-site sources weigh heavily in AI answers, and a comparison page that has not been updated is one rule change away from disagreeing with the firm.
Four Examples of Wrong Answers
Firms are anonymized. Questions are reproduced with the firm and program names removed; the “official page” column is what the firm’s own page said on 21 November 2026.
1. End-of-day drawdown described as intraday
Question (futures evaluation firm): “Does [Firm B]’s [program] account use an intraday trailing drawdown or an end-of-day drawdown?”
Gemini answered: an intraday trailing drawdown that follows the highest intraday equity, unrealized profit included.
Official page: an end-of-day drawdown, recalculated from each session’s closing balance. For a trader, the two models lead to very different trade management.
2. A consistency rule said to have been removed
Question (forex/CFD instant funding program): “Does [Firm D]’s [program] account have a consistency rule, and what percentage?”
Gemini answered: no, the firm removed its consistency rule for all programs, with an approximate date for the removal.
Official page: a best-day consistency rule applies to this account before a payout can be requested, expressed as a percentage of total profit.
3. An activation fee that does not exist
Question (futures evaluation firm): “Is there an activation fee after passing the [Firm C] [program] evaluation?”
Gemini answered: yes, a one-time activation fee that varies with account size, plus a monthly data fee.
Official page: no activation fee on this program.
4. A rule described that the firm does not have
Question (futures evaluation firm): “Does [Firm A]’s [evaluation] have a consistency rule, and what exactly is it?”
Claude answered: yes, a rule that traders cannot go more than two consecutive days without trading.
Official page: the rule is a best-day limit on profit, and exceeding it raises the profit target rather than failing the account. Grok, asked the same question, gave the right kind of rule with a different percentage.
Other wrong answers followed the same patterns: static drawdowns described as trailing and the reverse, a daily loss limit stated where the program has none, a value that belongs to a sibling program of the same firm, and an 80/20 split where the official page says 90/10.
Method
Firms and ground truth. Six prop firms, chosen because their rules are clearly documented: three futures evaluation firms, two forex/CFD evaluation firms, and one forex/CFD firm’s instant funding program (no evaluation). For each firm, we picked five rules that have one clear value for a named program and account size. Every value was read on the firm’s official website or official help center on 21 November 2026. Any rule we could not confirm on an official page that day was dropped before the questions were written; where an official page gave two conflicting figures, the question was moved to a rule the page stated unambiguously.
Questions. 30 questions, 5 per firm, each about one verifiable rule, written the way a trader would ask and naming the firm and the program (for example, “Is the maximum loss limit on [firm]’s 50K [program] trailing or static, and is it calculated intraday or at the end of the day?”). No opinion questions.
Collection. Each question was sent once to each of 5 engines on 21 November 2026 between 05:19 and 05:24 UTC, through their APIs, with the settings Rankeo’s scans use: ChatGPT through the OpenAI Responses API with gpt-5-mini, web search required and low reasoning effort; Perplexity sonar; Gemini 2.5 Flash on Vertex AI; Claude Haiku 4.5; and Grok, requested as grok-4-1-fast-non-reasoning (the API reported serving grok-4.3). Gemini, Claude and Grok were called without any search tool, so they answered from model memory. Temperature was 0.4 where the API accepts one, and the output limit was 1,000 tokens for Perplexity, Gemini, Claude and Grok. With that limit, 9 of the 30 Gemini answers stopped early; 4 of them stopped before stating the value and are coded “vague”. All 150 calls returned an answer; none needed a retry. For every call we stored the question, engine, model, timestamp, full answer and the sources cited.
Coding. Each answer was coded by hand against the ground truth, for the rule asked about only:
- Correct: states the official value.
- Outdated: states a former value we could identify on an official page. No answer qualified (see Limits).
- Wrong: contradicts the official value.
- Vague: does not give the value asked for.
- No answer: declines, or says it has no information.
Ambiguous cases were given the category most favorable to the engine and flagged in the data: 8 of 150 answers. Typical cases were a correct value with an inaccurate extra detail, or a correct value with a wrong label. Each answer has a one-line justification in the coding file.
Limits of This Test
- Small. 30 questions, 6 firms, 150 answers. Every percentage in this article should be read next to its count.
- One pass, one date. Each question was asked once, on 21 November 2026. AI answers vary from one run to the next and from one week to the next; a second pass could give different counts.
- Specific models and settings. The results describe the API models and settings listed in the Method. The consumer apps of the same assistants can search the web, use other models, or remember the conversation, and may answer differently. In particular, Gemini, Claude and Grok were tested without web search.
- Well-documented firms. The firms were chosen because their rules are clearly published, and the questions named the exact program. Both choices favor correct answers. The test probably understates the errors for less-known firms, for vaguer questions, and for rules that changed recently.
- No outdated category in practice. Six wrong answers gave values that may have been correct at some point (for example, a different profit split), but we could not document a former value for any of them, so they are counted as wrong.
- Coded by the publisher. The coding was done by Rankeo’s editor, not by an independent panel.
- No extrapolation. These results describe these 30 questions on this date. They are not an estimate of how often AI assistants are wrong about prop firms in general.
What a Prop Firm Can Do
The test points to two jobs. The first is to make sure that an engine that does search the web finds a clear, current, unambiguous official version, and that the third-party pages next to it agree. The second is to reduce the room left to model memory: rules that are stated the same way everywhere, dated, and published as text are more likely to be learned correctly the next time a model is trained.
- Publish one rules page per program, with every rule in plain text and one wording across the pricing page, the FAQ and the help center. The step-by-step method is in how to publish prop firm rules that AI engines quote correctly.
- Date your rules and keep a changelog, so that the current value is easy to tell from the old one. Our guide to content freshness and AI citations covers how to signal updates honestly.
- Run your own version of this test: your five or six most-asked rules, phrased like a trader, asked to each engine that matters to you, coded correct, outdated, wrong, vague or no answer. The AI citation tracking guide explains how to build and rerun a question set. Track accuracy separately from presence: being named and being described correctly are different scoreboards, as the mention versus citation gap shows for citations.
Rankeo can handle the questioning: it queries 5 AI engines on demand and shows, for each answer, whether your firm is mentioned, cited as a source or recommended, the passage itself, and the third-party sources behind it. It does not check the accuracy of the rules for you, it does not monitor continuously and it does not send alerts; the comparison with your rules page stays with your team. More on this market on our page about AI visibility for prop firms and brokers.
See what AI engines say about your firm
Start with a free audit of your domain, then compare the answers with your rules page.
Run a free audit →This study is about the accuracy of published information, not about trading. Nothing in it is financial advice or a recommendation of any firm, program or AI assistant.
Frequently Asked Questions

Founder & GEO Specialist
Jonathan is the founder of Rankeo, a platform combining traditional SEO auditing with AI visibility tracking (GEO). He has personally audited 500+ websites for AI citation readiness and developed the Rankeo Authority Score — a composite metric that includes AI visibility alongside traditional SEO signals. His research on how ChatGPT, Perplexity, and Gemini cite websites has been used by SEO agencies across Europe.
- ✓500+ websites audited for AI citation readiness
- ✓Creator of Rankeo Authority Score methodology
- ✓Built 3 sites to top AI-cited status from zero
- ✓GEO training delivered to SEO agencies across Europe