Methodology · current market version v1

A benchmark you can challenge, not a black-box promise

Rose-Brook captures how selected AI models answer a controlled set of hotel-choice prompts, then separates commercial preference from evidence quality. Every result is a dated sample, not a prediction of every answer a traveller may receive.

Check my hotel Product overview

Freeze the field

The user confirms the subject hotel and real competitors before the run.

Hold conditions constant

Every hotel is tested as the candidate with the same prompt pack, enabled providers and scoring version.

Preserve the evidence

Captured answers, provider identity, eligibility decisions and partial failures remain attached to the saved run.

Commercial preference

AI Choice Score

A 0–100 read of whether the captured answers appear ready to choose the hotel. At least three component types and 65% of the available weight are required; otherwise Rose-Brook reports insufficient evidence rather than forcing a number.

35%Strong recommendationsWhether an answer clearly recommends the hotel rather than merely mentioning it.

30%Head-to-head winsHow often the hotel wins comparable choice prompts against the confirmed field.

20%High-intent promptsPerformance where a guest is close to choosing or booking.

10%AI placementHow prominently the hotel appears when an assistant lists options.

5%Proof strengthWhether the answer contains enough support for the preference signal.

Knowledge and support

Evidence read

A separate 0–100 view of whether eligible answers recognise and describe the hotel accurately with adequate support. Preference-only prompts do not inflate this score.

22%RecognitionWhether the hotel appears in eligible discovery and visibility prompts.

24%Factual accuracyWhether captured property facts agree with the benchmark context.

18%Attribute coverageWhether relevant stay attributes and proof points are represented.

24%Evidence supportThe strength of supporting detail in the captured answer.

12%ConsistencyHow stable the evidence read is across eligible prompts.

What the models can access

Benchmark answers are captured without live web-search tools

The core benchmark calls the enabled OpenAI, Google and Anthropic provider APIs directly. It does not enable live browsing or web-search tools during those prompts. The result therefore reflects each model’s available knowledge plus the controlled prompt context at capture time. Provider behaviour can change between runs.

The separate action-plan workflow does use live web search. Its source URLs and search activity are saved so users can distinguish web evidence from benchmark model output.

Interpretation rule

Treat one run as a decision-quality snapshot. Treat movement as credible only when the hotel field, prompt pack, providers and scoring version remain comparable.

AI Choice Score

35%Strong recommendationsWhether an answer clearly recommends the hotel rather than merely mentioning it.

30%Head-to-head winsHow often the hotel wins comparable choice prompts against the confirmed field.

20%High-intent promptsPerformance where a guest is close to choosing or booking.

10%AI placementHow prominently the hotel appears when an assistant lists options.

5%Proof strengthWhether the answer contains enough support for the preference signal.

Evidence read

A separate 0–100 view of whether eligible answers recognise and describe the hotel accurately with adequate support. Preference-only prompts do not inflate this score.

22%RecognitionWhether the hotel appears in eligible discovery and visibility prompts.

24%Factual accuracyWhether captured property facts agree with the benchmark context.

18%Attribute coverageWhether relevant stay attributes and proof points are represented.

24%Evidence supportThe strength of supporting detail in the captured answer.

12%ConsistencyHow stable the evidence read is across eligible prompts.

Benchmark answers are captured without live web-search tools

The separate action-plan workflow does use live web search. Its source URLs and search activity are saved so users can distinguish web evidence from benchmark model output.