Embedding, Rerank API and Dify knowledge base access tutorial

If Dify can communicate normally, it does not mean that the knowledge base has been configured successfully. The chat model is responsible for generating answers, the Embedding model is responsible for converting text into vectors, and the Rerank model is responsible for re-ranking candidate documents. They are three different capabilities.

What do the three types of models do?

Don't fill in the chat model ID into the Embedding node, and don't think that model names are interchangeable because they are similar.

Check before accessing

1. Confirm Embedding and Rerank model IDs from the current model directory.
2. Confirm whether they use OpenAI Compatible endpoints or independent vendor protocols.
3. Confirm vector dimensions, single batch limit, and maximum input length.
4. Confirm whether the current Dify plugin supports custom Base URL and target model types.
5. Use a small amount of test documents to build a temporary knowledge base first, and do not rebuild the production index directly.

ViewCurrent models and prices, model type and API capabilities are subject to the console and API documentation.

Verify Embedding API first

When the target model explicitly supports OpenAI style Embedding endpoints, you can first run:

curl https://www.aifast.club/v1/embeddings \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "in the consoleEmbeddingmodelID",
    "input": ["What is the refund policy？", "How to apply for a corporate invoice？"]
  }'

Check:

• Returns whether the number of arrays is consistent with the input number.
• Whether each vector is a numeric array.
• Are all vector dimensions consistent?
• usage existence and verification of billing records.
• What error is returned when the input is empty, too long, or the batch size is too large.

If the current model is not an OpenAI style Embedding protocol, the dedicated endpoint given in the AIFast document should be used and cannot be forcibly called. /v1/embeddings。

Vector dimensions cannot be changed arbitrarily

After the knowledge base is built, the query vector and document vector must use compatible models and dimensions. Replacing the Embedding model usually means re-vectorizing the entire document.

Record before going online:

{
  "embedding_model": "MODEL_ID",
  "dimension": 0,
  "chunk_strategy": "documented-strategy",
  "index_version": "2026-07-15-v1"
}

Don't directly overwrite production indexes without version tags. Create a new index first, then switch after completing the regression.

Configure in Dify

Dify menu labels vary by version and plugin. Start from Settings / Model Provider:

1. Add a model provider that supports custom endpoints.
2. Fill in the independent test Key.
3. Fill in according to the plug-in requirements https://www.aifast.club/v1;If the plug-in automatically appends /v1, avoid according to request log /v1/v1。
4. Add Chat, Embedding and Rerank models respectively.
5. Create a temporary knowledge base and upload a small number of verifiable documents.
6. Run the fixed problem set after vectorization is complete.
7. Enable Rerank again and compare the search results before and after enabling it.

Do not change chunking, Embedding, recall quantity, Rerank and prompt words at the same time, otherwise it will be impossible to locate the cause after the result changes.

How to set up chunking

There is no fixed chunk size that fits all documents. It is recommended to start with the business structure:

• FAQ: One question and one answer as a block.
• Product manual: divided into titles and sections.
• Contracts and Institutions: Clause numbers and context reserved.
• Code: Split by function, class or module boundaries.

Each block retains the document ID, title, chapter, update time, and permissions tags. The search results must be able to return to the original text, rather than just a piece of text that cannot be located.

How to accept Rerank

The input to Rerank is usually a user query and a set of candidate documents, and the output is a relevance score or reranking result. It cannot retrieve documents that were completely missed during the recall phase.

Create a test set that contains at least the following situations:

1. The answer is clear and there is only one correct chapter.
2. Several chapters have similar content.
3. Questions contain abbreviations, synonyms, or product models.
4. There is no answer in the documentation.
5. User does not have permission to access some candidate documents.

Also compare Recall@K, correct document ranking, final answer citations, and no-answer rejections. Looking only at the fluency of the final answer can mask incorrect retrievals.

Common mistakes

Dialogue succeeds but knowledge base keeps failing

Check whether the Embedding model, endpoint and model type are configured individually. The fact that the chat key is available does not mean that the current account has the target vector model permissions.

dimension mismatch

Querying and building databases use different models or different dimensions. Restore the original model, or re-vectorize with the new model and switch index versions.

model not found

from /v1/models Or copy the exact model ID from the console and confirm that the Dify node type is consistent with the model capabilities.

The search result contains content but the answer is wrong

Raw recall, post-Rerank ranking, cue word citations, and final generation were examined separately. Don’t just tweak the chat model.

Batch database creation appears 429

Reduce concurrency and batch size, use capped backoff, save completed document IDs. Only failed shards are processed during retries to avoid vectorizing the entire knowledge base repeatedly.

Check before going online

• Chat, Embedding, Rerank use independent and correct model types.
• Vector dimensions, chunking strategies, and index versions are documented.
• Permission filtering remains in effect before reranking and building.
• The fixed question set contains similar documents, no answers, and permission scenarios.
• Batch database creation supports breakpoints, rate limiting and failure retries.
• Replacing the Embedding model creates new indexes and regression validation.
• The log does not save the complete API Key and sensitive original text.

Continue readingDify basic accessandModel selection method, first make the retrieval evidence correct, and then optimize the final answer style.