Let me conclude first: The total score of the test report is "API comprehensive compatibility", not the model intelligence score, nor the official identity probability. The reading order should be to look at the model statement first, then the failure evidence, and finally the total score.
Open the detection toolFinally, it is recommended to choose standard mode. Standard mode adds SSE, tool invocation, output style, and knowledge boundary sampling, making it more suitable for pre-access checks.
In what order should the reports be read?
1. Checked address and request model
First make sure that the Base URL and model ID are filled in correctly. Common errors include appending repeatedly /v1/v1, treat the display name as the model ID, or fill in /chat/completions specific path.
2. Request model and response model
Consistent means that the API declarations in this round are consistent; different means that there may be differences in aliases, version resolution, or routing. Neither can prove the underlying identity alone, and one needs to continue to look at capabilities and metadata.
3. Sub-item status
| Status | meaning | Recommended action |
|---|---|---|
| Pass | This probe met expectations | Join the business question set to continue verification |
| Doubtful | There are results but there are structural or consistency issues | Review the evidence and retest |
| failed | Key protocols or capabilities not met | Check the configuration first, then contact the service provider |
| observe | Weak signal, not counted in compatibility | Only used for multi-round trend comparison |
| No results | The upstream did not return interpretable content | Check permissions, rate limits, timeouts, and protocol coverage |
| Not executed | This item is not running in the current mode | Switch to standard mode when needed |
4. Overall score and business impact
A high score indicates a more complete protocol and key probes, but does not mean a stable production environment. A low score does not necessarily mean that the model is fake, it may be that the target endpoint only implements a partially compatible protocol.
The weight should be determined based on the business: ordinary Q&A pays more attention to basic protocols and streaming output; Agent pays more attention to tool invocation; cost-sensitive projects pay more attention usage Integrity; production systems must also be individually tested for latency, concurrency, error rates, and rate limiting.
What do the ten tests indicate?
- Certifications and model list: Whether the Bearer Key is certified,
/modelsWhether a recognizable directory is returned, and whether the selected model ID is visible; a restricted directory does not mean that the model is fake. - Protocol layer compliance: Whether the Chat Completions status code, response JSON, and key fields are recognized.
- Fixed instructions to follow: Can a random string be returned as-is, independently observing that the constraints are followed.
- Metadata fingerprint:
id、object、created、modeland request ID is complete. - Billing Token field: Whether the input, output and total tokens are available and arithmetically consistent.
- Output style characteristics: Weak features such as paragraphs, headings, lists, and code fences are only used for comparison.
- Knowledge cutoff boundaries: The knowledge boundaries described by the model are weak signals that can easily be rewritten.
- R1 dynamic questions: Capability sampling using random parameters and nonces.
- SSE streaming output: Streaming events, incremental content, and closing tags are compatible.
- Tool call: Whether the function name, JSON parameters, and random values are returned as required.
How to fairly compare two API gateways
- Create temporary quota API Keys for both sites.
- Select the same model series and the same detection mode.
- Try to execute it at a similar time and in the same network environment.
- Repeat at least three rounds for each endpoint.
- Record model declaration, failure items, tokens, time-consuming and business problem set results.
- Compare evidence, don't just compare total scores once.
If two service providers use different model aliases, you should first confirm the actual mapping with their respective consoles. If it cannot be confirmed, mark it as unknown in the report and do not complete the conclusion on your own.
Under what circumstances should access be stopped immediately?
- API keys or sensitive information may be logged, exposed, or transmitted over non-HTTPS.
- Authentication, model routing, or accounting data continues to be abnormal and the service provider cannot explain it.
- Business-essential tool calls, streaming output, or structured results repeatedly fail.
- Results continue to drift under the same conditions and lack traceable request IDs or after-sales responses.
These conditions focus on usability, transparency, and security boundaries, and do not rely on "100% authenticity."
Reporting cannot replace anything
Test reports cannot replace manufacturer signatures, supply chain audits, contracts and privacy clauses, nor can they predict future stability. Production rollout also requires continuous monitoring of delays, error rates, throttling, routing changes, and billing consistency.
Next step
Read firstRepeatable detection method for model quality degradation, and then supplement the report with a set of fixed business questions. When you find inconsistencies in model declarations, checkModel impersonation and route substitution checks。