Build a local knowledge assistant over operating procedures

Choose a controlled source collection

Start with approved procedures for one process. Give each document an identifier, revision, effective date, owner, and access rule. Exclude drafts, personal folders, and withdrawn instructions. A local model does not make an uncontrolled knowledge collection authoritative.

Keep the original document available beside the answer. The assistant should reduce lookup effort while making verification easier. It should not become a second, unowned copy of the organisation’s operating policy.

Separate the pipeline into stages

A practical retrieval-augmented generation pipeline has four independently testable stages: ingestion, retrieval, answer generation, and verification. Store source references with every chunk. Apply access filters during retrieval, before text enters the model prompt.

approved document → versioned passage → permission-filtered retrieval
                  → bounded prompt → cited answer or unresolved state

Use known identifiers, headings, and semantic retrieval together where appropriate. Chunk by meaningful sections rather than fixed character counts alone. Keep tables with the labels needed to interpret them. A passage containing a number but not its unit is a poor source for a reliable answer.

Design the response contract

Ask the model for an answer, source identifiers, and an unresolved reason when the evidence is insufficient. Validate that each cited identifier was actually retrieved for the current user. The model must not invent a document reference that looks credible.

{
  "answer": "Review the current handover checklist before release.",
  "sources": ["SYNTHETIC-SOP-07:revision-3:section-2"],
  "unresolved": null
}

This is an illustrative response shape. The application needs to check source existence and permissions, and a reviewer still needs to establish whether the answer is supported by the cited passage.

Evaluate retrieval before generation

Create questions with known answer passages. Measure whether the required source appears in the retrieved set. Then assess answer correctness, citation support, and abstention. If the correct passage is absent, increasing model size may simply generate a more fluent unsupported answer.

Include obsolete revisions, similarly named procedures, inaccessible documents, contradictory sources, and questions the corpus cannot answer. Treat instructions embedded in source documents as untrusted text. They must not alter tool permissions or the answer policy.

Keep updates inexpensive and recoverable

Track checksums and re-index only changed material. Remove withdrawn sources from retrieval and invalidate dependent cached answers. Keep the prior index available until the new one passes regression tests.

Record the model, embedding model, chunking settings, and retrieval configuration as a versioned release. A change to any of these can alter answers. Model weights do not need fine-tuning every time a procedure changes; the source pipeline should be designed for normal document updates.

Operate within the access boundary

Local hosting can keep inference on the workstation or network, but access controls, logs, backups, and client applications still need attention. Avoid recording unnecessary sensitive text in diagnostics. Restrict who can change the source collection and who can administer the model service.

Budget for document-owner time, ingestion fixes, retrieval evaluation, infrastructure, and answer review. Use accepted, verified answers as the denominator when comparing options.

References

• Ollama API
• Ollama FAQ
• Work-instruction implementation playbook