RAG vs Fine-Tuning for Arabic Enterprise AI

Saudi organisations building Arabic AI systems face the same first decision: retrieve or fine-tune. Choosing wrong wastes months and a large budget. The answer is usually clearer than the debate suggests.

The two approaches in one paragraph each

Retrieval-augmented generation (RAG) keeps your knowledge in a searchable store. When a question arrives, the system finds the relevant documents and passes them to the model as context. The model's weights never change; you are supplying it with the right reference material at the moment of answering.

Fine-tuning adjusts the model's weights by training on your examples. The knowledge and behaviour become part of the model. It is slower and more expensive to change, but produces consistent style, format and domain-specific behaviour without needing retrieved context every time.

When RAG is the right answer

For the large majority of Saudi enterprise use cases we scope, RAG is correct. It suits situations where the knowledge changes, where citations matter, and where access control varies by user.

• Internal knowledge assistants over policies, procedures and contracts
• Customer support answering from a product or service knowledge base
• Document search across regulatory or legal corpora
• Anything where the source of an answer must be auditable
• Anything where content updates weekly or monthly
• Anything where different users may see different documents

The decisive advantage is that updating knowledge means updating a document, not retraining a model. In a regulatory environment where guidance changes, that difference is operational rather than theoretical.

When fine-tuning earns its cost

• Enforcing a consistent output format the base model keeps drifting from
• Adopting a specific institutional tone across thousands of generations
• Domain language the base model handles poorly — specialised Arabic legal or medical terminology
• Classification tasks where a smaller fine-tuned model is faster and cheaper than a large general one
• Reducing prompt length materially at very high request volumes

Note what is absent from that list: adding facts. Fine-tuning to teach a model your company's facts is the most common and most expensive mistake in enterprise AI. It produces a model that states outdated information fluently and without citation, which is worse than one that says it does not know.

Fine-tune for behaviour. Retrieve for facts. Most failed enterprise AI projects inverted that.

Arabic-specific considerations

Retrieval quality is the hard part

Arabic retrieval is harder than English retrieval for structural reasons. Rich morphology means one concept appears in many surface forms. Diacritics are usually absent in real documents. Dialect varies between formal written Arabic and how users actually phrase questions. And many enterprise Arabic documents exist only as scanned PDFs.

1. Use embedding models with genuine multilingual Arabic training, not English models with token coverage
2. Normalise text — unify alef and hamza forms, standardise ta marbuta and ya, strip diacritics consistently across both index and query
3. Run hybrid search combining dense vectors with BM25 keyword matching; Arabic keyword search catches exact terms that embeddings miss
4. Use Arabic-capable OCR for scanned documents and review the output quality before indexing
5. Chunk on semantic boundaries rather than fixed character counts, which cut Arabic sentences awkwardly
6. Add a reranking step — it improves Arabic retrieval more than it typically does in English

Model choice

Several open-weight families now handle Arabic competently, including Qwen, Llama and regionally tuned derivatives, alongside commercial models with strong Arabic performance. Evaluate on your own documents and your own questions rather than on published benchmarks, which rarely reflect enterprise Arabic in a specific domain.

Data sovereignty changes the calculation

Many Saudi organisations cannot send data to third-party APIs — government entities, financial institutions, healthcare providers and anyone handling data under residency obligations. That constraint should be established before any architecture discussion, because it eliminates options.

Self-hosting open-weight models on in-Kingdom infrastructure is entirely practical now. A well-chosen mid-sized model on a modest GPU server handles typical internal assistant workloads comfortably, and inference never leaves your network. The trade-off is that you own operations — deployment, monitoring, scaling and updates.

A practical decision framework

1. Does the data have to stay inside your infrastructure? If yes, plan for self-hosted open-weight models from the start.
2. Is the task about facts or behaviour? Facts point to RAG. Consistent format, tone or classification points to fine-tuning.
3. How often does the knowledge change? Weekly or monthly changes make retraining impractical.
4. Do answers need citations? If yes, RAG is effectively mandatory — fine-tuned knowledge cannot be attributed to a source.
5. Do different users see different documents? Access control is natural in retrieval and impossible in weights.
6. Do you have a few thousand high-quality labelled examples? If not, fine-tuning has nothing to learn from.

In practice, mature systems often use both: RAG for knowledge, plus a lightly fine-tuned or heavily prompted model for consistent output format. Start with retrieval, measure where it falls short, and add fine-tuning only against a specific measured deficiency.

How to start without wasting a quarter

Pick one narrow use case with a measurable baseline — for example, how long staff currently take to find a policy answer. Build a RAG prototype over a few hundred real documents. Test with fifty real questions written by the people who will use it. Measure answer accuracy and retrieval precision separately, because most failures are retrieval failures rather than generation failures.

That prototype takes weeks, not quarters, and it tells you more about your data quality than any strategy document will. Data quality, not model choice, is what determines the outcome of most enterprise AI projects in our experience.