Import a website

Import a website to give the Assistant its pages as Grounding Knowledge. Each page keeps its address, so the Embedded Assistant can link visitors to it and list it under an answer as a source.

The site's domain must be one of the Assistant's saved Origin Domains: the importer reads no other host. See Set an Assistant's Origin Domains.

Import a site

  1. In AI Assistants, open the Assistant.

  2. On the Configure tab, go to Knowledge Base > Websites.

  3. In the address field, enter the site's address, or the address of a folder inside it such as https://www.example.com/help/. For the first website, the field already holds your first Origin Domain.

  4. Click Import.

The website shows "Importing… the pages appear below once they are read." When it shows how many pages it imported, the pages are being indexed, and they go live when the Indexing banner clears. The website also appears under Uploaded as one row with its page count. Importing doesn't need Save Changes.

An Assistant imports up to 10 websites, such as a marketing site and a help set, or several help sets on one host. It runs one import at a time: while one runs, the other websites' buttons are unavailable.

What an import reads

Rule

What it means

The sitemap is the list

The importer reads the pages the site's sitemap lists. It looks for sitemap.xml in the address's folder, then the Sitemap: entries in robots.txt, then /sitemap.xml, and follows sitemap indexes, including one that lists the parcels of a Reverb 2.0 help.

No sitemap

The importer starts at the address and follows its links, on the same site and inside the folder. Pages that are mostly links, such as a table of contents, are followed but not imported.

The folder sets the scope

Only pages under the address's folder are imported: https://example.com/help/ takes the help and leaves the rest of the site.

www makes no difference

example.com and www.example.com are one site. Entering a site you already import imports it again rather than adding it twice.

Page cap

One import takes up to 2,000 pages. A larger site imports its first 2,000 and says so; import folders inside it as separate websites to cover more.

What a page gives

Its main content: headings, text, lists, tables, code, links made absolute, and images' alt text.

What an import leaves out

  • Pages on other hosts, including other subdomains of an Origin Domain. Import each host as its own website.

  • Pages the site's robots.txt disallows for the importer, which identifies itself as WebWorksPlatformBot.

  • Files that aren't web pages, such as PDFs, images and downloads. Upload those instead.

  • Each page's navigation, header, footer and sidebars, its scripts and its form controls.

  • Text a page adds with JavaScript after it loads. The importer reads each page as the server sends it.

  • Pages with almost no text, and pages too large to import.

  • Pages that don't answer, and pages that redirect to a host outside the Origin Domains.

Keep imported pages current

While the Assistant is in deployment, the Platform imports each of its websites again about every 7 days. An Assistant that isn't in deployment is refreshed only when you ask. To pick up a change sooner, click Refresh beside the website.

A refresh reads again only the pages that changed. It adds pages that have appeared in the sitemap and drops pages that are gone. A page that can't be reached during a refresh keeps the copy from the last import.

If a refresh fails, the website shows "Import failed:" and the reason, its button reads Retry, and the Assistant keeps answering from the pages it had. When runs keep failing, the organization's billing email gets a notice. See Knowledge Base problems.

Remove a website

Click Remove this website (the trash icon) beside the website under Websites, or on its row under Uploaded. The website is removed at once, without Save Changes, and its pages leave the Knowledge Base when indexing finishes.

Last modified date: 11/02/2026