Adding AI to your product is no longer optional in many markets — but how you add it matters. Hosted APIs (OpenAI, Google, Anthropic, etc.) get you to market fast. Fine-tuning gives you control over tone, domain, and cost at scale. Custom models are for rare cases where neither fits.
From our experience integrating AI into SaaS, marketplaces, and internal tools, the biggest mistakes come from choosing the wrong pattern: over-investing in custom models when an API would do, or locking into an API when fine-tuning would have saved money and improved quality.
This guide walks you through the three main integration patterns, when to use each, and what to watch out for.
The Three Main AI Integration Patterns
Before diving into tradeoffs, it helps to be clear on what each pattern means.
1. Hosted APIs (Prompt-Based)
You send prompts (and optionally images, documents, or structured inputs) to a provider's API. They run the model and return text, structured JSON, or embeddings. You pay per token or per request.
Examples: OpenAI GPT-4 and GPT-4o, Google Gemini, Anthropic Claude, Cohere, together.ai, and similar. You do not train or host the model; you only call it.
- Fastest to integrate — often a few days to a working feature
- No infra or ML team required
- Provider handles updates, scaling, and compliance basics
- Cost scales with usage; at high volume, per-token cost can add up
- You have little control over model behavior beyond prompts and parameters
2. Fine-Tuning (Or Instruction Tuning)
You start from a base model (yours or a provider's) and train it further on your own data or examples. The result is a model that better matches your domain, style, or constraints. You may run it yourself or use a managed fine-tuning service.
Examples: OpenAI fine-tuning for GPT-4o mini, open-weight models (Llama, Mistral) fine-tuned on your docs or chat logs, or domain-specific models offered by vendors.
- Better quality and consistency for your specific use case
- Often lower per-token cost at scale than raw API calls
- You can reduce prompt size and latency by baking knowledge into the model
- Requires curated data, some ML workflow, and ongoing evaluation
- Still depends on a base model; you are not building from scratch
3. Custom Models (Train or Build From Scratch)
You train or commission a model tailored to your problem — different architecture, data, or objective. This includes small bespoke models for classification or retrieval, or large models trained on proprietary data.
Examples: In-house models for medical or legal domains, proprietary recommendation or ranking models, or vertical-specific models that are not general-purpose chat.
- Maximum control over behavior, data, and IP
- Can be the only option for highly regulated or proprietary domains
- Highest cost and time: data pipelines, training, evaluation, and ops
- Only justified when APIs and fine-tuning cannot meet your requirements
Bottom line
Most products should start with hosted APIs. Move to fine-tuning when you have clear quality or cost gains and the data to support it. Consider custom models only when APIs and fine-tuning cannot meet compliance, IP, or performance needs.
When to Use Hosted APIs
Hosted APIs are the default choice for most teams. Use them when:
- You need to ship an AI feature quickly and validate demand
- Your use case is well served by general-purpose language or vision models
- Volume is low to medium, or you are okay with per-token pricing
- You want to avoid ML ops, model updates, and compliance burden
Typical Use Cases
Chatbots and support automation, content summarization, code assistance, image generation or editing, semantic search via embeddings, and light personalization (e.g. dynamic copy). These rarely require fine-tuning in v1.
What to Watch Out For
Rate limits and quotas can bite at launch or during spikes. Design for retries, fallbacks, and optional queuing. Cost can grow fast with volume — monitor usage and set alerts. Prompt injection and output consistency are your responsibility; invest in prompt design, output validation, and guardrails.
Rule of thumb
If you can describe the behavior you want in a prompt and the API gets you 80% there, ship with the API first. Optimize later.
When to Use Fine-Tuning
Fine-tuning becomes attractive when you have:
- Enough high-quality examples (hundreds to thousands, depending on task)
- A clear gap between API output and what you need (tone, terminology, format, or accuracy)
- Enough usage that per-token savings or quality gains justify the effort
- Capacity to maintain datasets, run training, and evaluate outputs
Typical Use Cases
Domain-specific Q&A or support (e.g. legal, medical, internal docs), consistent brand voice in generated copy, structured output that must follow a strict schema, and cost reduction at scale by using a smaller fine-tuned model instead of a large API.
What to Watch Out For
Garbage in, garbage out. Fine-tuning amplifies biases and errors in your data. Curate and review training data. Plan for iteration: you may need several rounds of data and evaluation. If you use a provider's fine-tuning, lock in and pricing can become a concern; document your exit path (e.g. export or open-weight fallback).
Takeaway
Fine-tuning is an optimization step. Do it when you have evidence that the API is the bottleneck — not before.
When to Consider Custom Models
Custom models (train or build from scratch) make sense only when:
- Regulation or IP requires data and model to stay in-house or in a specific jurisdiction
- The task is not well served by general-purpose LMs (e.g. specialized ranking, fraud, or scientific models)
- You have large, proprietary datasets and the budget and team to build and operate the system
Typical Use Cases
Vertical-specific models in healthcare, finance, or legal where APIs are not compliant or accurate enough. Recommendation, ranking, or fraud models that are not "chat" at all. Or long-term differentiation where the model itself is the product.
What to Watch Out For
Custom models are a major commitment. Timeline is months, not weeks. You need data engineering, ML engineering, evaluation, and ongoing ops. Before going this route, explicitly confirm that fine-tuning or API + guardrails cannot meet your requirements.
Key principle
Custom models are for when the integration pattern is the product — not for "we want our own AI" without a concrete technical or regulatory reason.
Comparison at a Glance
Hosted API: fastest to ship, no ML ownership, cost scales with usage, limited control. Fine-tuning: better quality/cost for your domain, needs data and iteration, some lock-in. Custom: full control and IP, highest cost and time, only when necessary.
How We Choose at Vertecs
For most client products we start with a hosted API and strong prompt design, output validation, and error handling. We add fine-tuning when we see repeated quality issues or cost pressure and the client has (or can create) the data. We recommend custom models only when compliance, IP, or task fit demands it and the client is prepared for the investment.
Getting the integration pattern right early avoids costly rework and keeps the door open to better options as your product and data mature.
