Indexing
Indexing is how Context MCP Studio turns your sources into searchable, citable context. It runs in the background — not inside a single browser request — so closing the dashboard, a timeout, or a deploy mid-run cannot wipe a working index.
| Audience | MCP operators, content owners, and developers troubleshooting source quality |
|---|---|
| Prerequisites | At least one source connected to an MCP server |
What happens during indexing
- MCP Studio queues the source and starts working on it immediately.
- It discovers the pages or files the source contains.
- It fetches each one, retrying transient failures, and skips anything that has not changed since the last run.
- It extracts the meaningful content, splits it into passages that preserve headings and code blocks intact, and prepares them for search.
- New content is written alongside the existing index. Stale pages are only removed once a run finishes successfully, so a failed run leaves your last good index fully searchable.
Large sources are processed incrementally and continue in the background until complete. You do not need to keep the dashboard open.
Indexing status
Each source shows one of these statuses:
| Status | Meaning |
|---|---|
| Pending | Source has been added; work is queued |
| Indexing (dashboard may show Crawling) | Content is being discovered, fetched, and indexed. A percent and page counts appear on the dashboard. |
| Complete | All content has been indexed and is searchable |
| Error | Something went wrong — see the inline error reason below the source |
While a source is indexing you can still click Refresh / Restart indexing. That cancels the current run and starts a fresh one. If a run stops reporting progress, the dashboard marks it stalled and it is automatically retried or failed rather than being left stuck.
When a source fails
You do not have to watch the dashboard. When a source fails to index, we email you with what happened and what to do about it. You get one email per source per day, so refreshing a source that keeps failing will not fill your inbox, and other sources on the same server keep working and stay searchable.
The server's page shows the same explanation in a banner below the failed source, sometimes with a button that performs the fix.
Error reasons
| Cause | What it means | Action |
|---|---|---|
| Repository access | Private GitHub repo and we have no valid token for your account | Click Link GitHub next to the source |
| Repository not found | GitHub returned 404 with your account linked, so it was renamed, deleted, or your access was removed | Verify the URL, or re-grant organization access |
| GitHub throttling | GitHub is temporarily limiting how fast we can read | Nothing. Indexing resumes on its own |
| No response | The fetch timed out or the site was unreachable | Nothing yet. This retries automatically |
| No indexable text | Pages returned almost no extractable text, usually a login-gated site or one that builds its content with JavaScript | Point at a page whose text is visible without signing in, or upload the files instead |
| MCP server refused | A connected MCP endpoint needs a key, or rejected the one it was given | Update the key on the source, then refresh |
| MCP server unreachable | A connected MCP endpoint did not respond | Check the endpoint URL, then refresh |
| Not an MCP server | The URL answered, but not as an MCP server | Check the endpoint URL. Most hosted servers end in /mcp |
| Uploaded file missing | The uploaded file is no longer stored | Remove the source and upload the file again |
| Uploaded file empty | No text could be read from the file | Remove the source and upload a version that contains text |
| Indexing stalled | The job stopped reporting progress and was reset | Click Restart indexing |
| Index full | Our index storage is full, so this source could not be written | Free space by removing a source or server you no longer use |
| Over your plan | The source is larger than your plan allows | Upgrade, or remove another source from this server |
A login-gated or JavaScript-rendered site is the most common failure, and no amount of refreshing fixes it, because there is no text to read without signing in. If your documentation is not publicly readable, upload the files directly and skip the crawler.
Typical indexing times
- 1–3 sources: Usually 1–6 minutes
- Large docs sites: Can take longer and will continue in the background
- GitHub repos: Often faster for markdown-first repos, longer for broad code repositories
- Unchanged sources on daily refresh: Much faster, because unchanged content is not reprocessed
Using your server during indexing
Your MCP server is usable immediately after deployment, even while sources are still indexing. During this period it falls back to fetching content live when the index does not yet have a match — queries still work, they are just slightly slower.
Once indexing completes, queries are served from the prepared index and are considerably faster.
Refreshing sources
You can refresh sources in two ways:
- Manual refresh: Click Refresh (or Restart indexing) next to any source on the dashboard. This is available even while a source is already indexing.
- Auto-refresh: Sources that have not been indexed recently are re-queued daily at 7:00 AM Central US time. Content that has not changed is detected and skipped.
Closing the browser does not stop a refresh — it continues server-side.
Page limits
- Website and GitHub sources are limited to 5,000 pages (or files) per source
- Very large files and non-text assets are skipped
- If a source is larger than the cap, use a more specific docs URL or the underlying GitHub docs repo
Citations
Answers cite the page they came from, so any claim can be traced back to your documentation. Citations retain section headings where the source provides them, and MCP Studio favors your current documentation when a source publishes several versions side by side.
Deep-link behavior by source type
- Web/docs: Citations resolve to section anchors when available.
- GitHub: Citations can resolve to file line ranges for code-level provenance.
- PDF: Citations include page fragments (
#page=N) when page-level context is available. - Federated MCP sources: Citations include MCP endpoint references and remote tool usage context. These sources are queried at runtime, not crawled.
GitHub access
For GitHub sources, MCP Studio uses the server owner's linked GitHub account to read the repository. Before creating the source it verifies that the repository is readable with that account, so a private repo you cannot access fails immediately with an actionable Link GitHub button rather than silently producing an empty source.
The OAuth scope requested is read:user user:email repo. The repo scope grants read access to private repositories owned by, or shared with, the user. Tokens are stored server-side and are never exposed to the browser.