How to scrape Bing Copilot Search (Bing AI Mode) in 2026

The page goes by several names. Microsoft calls it Copilot Search in Bing and serves it at bing.com/copilotsearch. Microsoft is testing the label “AI Mode” for the Copilot tab in Bing, cloro sells its endpoint as Bing AI Mode, and other vendors sell it as Bing Copilot.

All three names point at the same page, so when you scrape Bing Copilot, bing.com/copilotsearch is what you load. The Copilot app at copilot.microsoft.com is a separate chat product with its own transport and session handling. This guide does not cover it; our Copilot app scraping guide does.

Everything below comes from 12 Copilot Search pages we scraped on 2026-11-07 through cloro’s Bing AI Mode API, plus a timing run of 10 more prompts on 2026-11-08. The 12 prompts covered shopping, local, recipe, news, health, finance, sports, travel and how-to queries. We stored the outer page and the answer iframe for each one, and every selector in this post comes from those files.

Copilot Search is a search page, not the Copilot app

The difference decides how you scrape Bing Copilot. The app streams its answer over a socket. Copilot Search writes its answer into a normal web page, inside an iframe, and you read it from the DOM.

	Copilot Search in Bing	Copilot app
URL	bing.com/copilotsearch?q=...	copilot.microsoft.com
Where the prompt goes	The q parameter of the URL	A chat message
Where the answer arrives	An iframe on a Bing results page	WebSocket frames
What you parse	The frame’s DOM	The event stream
Guide	This post	How to scrape Copilot

The prompt rides in the URL, so very long prompts run into URL length limits. cloro’s API caps a Bing AI Mode prompt at 3,500 characters after URL encoding for that reason, and a DIY scraper should check the encoded length before it opens the page.

Copilot Search answers a query with a short essay and a handful of cited pages. If you track AI visibility, you need to know which prompts cite your pages and which cite a competitor. Microsoft offers no API for this page (see the FAQ below), so the rendered page is the only source. Once you can scrape Bing Copilot reliably, Copilot rank tracking is the next step.

What a Copilot Search page contains

Take the prompt “best italian restaurants in chicago”. The page opens with a reasoning panel: Searching for "best italian restaurants in chicago", then “Reading sources and understanding” above six site names and a “+3 more” button. Under it, the answer starts with a one-sentence summary, breaks into sections with headings, and carries inline chips such as “The Infatuation +2” or “Bing”. A table lists restaurants, and a row of source cards closes the answer.

All 12 pages had this same layout, including the shopping, local and video prompts. Bing showed no product, place, map, video or ad module in any of them. So if you scrape Bing Copilot for one query type, the same parser works for every type we tried.

Page element	Where it sits in the frame	Seen in our 12 samples	cloro API field
Search query	h2.b_ts_ls_hdrtext	12/12, equal to the prompt every time	none
“Reading sources” list, with “+N more”	ul.b_ts_sources > li.b_ts_source_pill	12/12, 6 to 11 pages each	sources (same order, folded pages included)
Inline citation chips (1, The Infatuation +2)	div.gs_sup_cit > a	11/12 (health had none), 242 in total	citationPills
Source cards below the answer	div.gs_cit[data-citnumb]	12/12, 2 or 3 cards each	already in sources
Per-section “N Sources” groups	.gs_cit_stry_preview_txt	12/12	none (same pages as the chips)
Summary sentence, headings, lists	.gs_heroTextHeader, h3, h4	12/12	text, markdown
Tables	table inside the answer	6/12	markdown only
Follow-up question	the last block of the answer	12/12 end with an offer, 6 under a heading	markdown only

Local answers change how you count citations. In the local-restaurants sample, Bing cited its own local listings with chips labelled “Bing”: 18 of the 42 citation pills in that answer point at bing.com, and 5 of its 14 sources are Bing search pages, 4 of them local listings (filters=local_ypid). If you rank domains by citations, decide up front whether bing.com counts.

Bing also wrote the restaurants’ addresses and phone numbers as plain text, labelled “Places API result”, with no structured place object behind them.

Almost everything you want is in data-* attributes and short, readable class names. Tables and the follow-up question exist only as rendered HTML, and you convert them yourself if you need them.

How to scrape Microsoft Copilot answers programmatically from Bing with Python

To scrape Bing Copilot, the code below runs five steps for each prompt:

1. Open bing.com/copilotsearch?q= with the prompt in the URL.
2. Find the answer iframe and wait until the answer stops changing.
3. Read the frame’s DOM for the query, the answer text and the tables.
4. Collect the sources from the “Reading sources” list, folded pages included.
5. Map each citation chip to its sources.

Step 1: open the page and wait for the answer to finish

In the outer page, the answer frame sits at #b_copilot_search li.ca_turn div.frame_cont iframe, and its aria-label is the query followed by “result container”. In the pages we stored, the iframe carries the answer document in its srcdoc attribute. In a live browser, read the frame through Playwright instead of parsing the attribute. Scripts inside the frame write the answer one token at a time: in the stored pages, every word is its own span.gs_tkn.

We found no “done” flag in the DOM. Finished pages still say “Reading sources and understanding” in the reasoning panel. So the code below polls the size of the answer container and treats three quiet seconds as the end.

import asyncio
from urllib.parse import quote_plus

from playwright.async_api import async_playwright

COPILOT_SEARCH = "https://www.bing.com/copilotsearch?q="
FRAME = "#b_copilot_search .frame_cont iframe"
SIZE_JS = """() => {
  const el = document.querySelector('.b_genserp_container');
  return el ? el.innerHTML.length : 0;
}"""


async def wait_until_stable(frame, quiet_s=3.0, timeout_s=90):
    last, quiet = -1, 0.0
    for _ in range(int(timeout_s / 0.5)):
        size = await frame.evaluate(SIZE_JS)
        if size > 0 and size == last:
            quiet += 0.5
            if quiet >= quiet_s:
                return
        else:
            quiet = 0.0
        last = size
        await asyncio.sleep(0.5)
    raise TimeoutError("Copilot Search answer did not settle")

Step 2: read sources, chips, cards and text

Run the extraction inside the frame, so one round trip returns everything. The “Reading sources” items behind “+N more” carry the class b_hide. They are in the DOM with the same attributes as the visible ones, so the code keeps them and marks them as folded.

EXTRACT_JS = """() => {
  const all = (sel, root = document) => Array.from(root.querySelectorAll(sel));
  const answer = document.querySelector('.b_genserp_container');

  const query = document.querySelector('h2.b_ts_ls_hdrtext')?.innerText.trim();

  const sources = all('li.b_ts_source_pill').map((li, i) => ({
    position: i + 1,
    title: li.dataset.title,
    url: li.dataset.url,
    site: li.dataset.domain,
    snippet: li.dataset.snippet,
    folded: li.classList.contains('b_hide'),
  }));

  const cards = all('.gs_cit[data-citnumb]', answer).map(c => ({
    num: c.dataset.citnumb,
    url: c.dataset.url,
    site: c.dataset.displayname,
  }));

  const chips = all('.gs_sup_cit > a', answer).map(a => {
    const dom = a.querySelector('.gs_sm_cit_dom');
    return {
      href: a.href,
      label: dom ? Array.from(dom.children).map(n => n.innerText).join(' ') : '',
      sups: (dom?.dataset.sups || '').split(',').filter(Boolean),
    };
  });

  const hero = answer.querySelector('.gs_heroTextHeader')?.innerText.trim();

  // Remove chips and source widgets so the text and tables read cleanly.
  all('.gs_sup_cit, .gs_cit_cont, .gs_cit_stry_preview', answer).forEach(n => n.remove());

  const tables = all('table', answer).map(t =>
    Array.from(t.rows).map(r => Array.from(r.cells).map(c => c.innerText.trim())));

  return { query, sources, cards, chips, hero, tables, text: answer.innerText.trim() };
}"""

Step 3: resolve each chip to its URLs

A chip’s link holds one URL, but the chip can stand for several sources, and a scraper that reads only the link loses the rest. The data-sups attribute on .gs_sm_cit_dom lists the citation numbers the chip covers, and the badge shows how many there are. Across our samples, a chip labelled “1” always covered one source, “2” covered two, and “The Infatuation +2” covered three.

To turn numbers into URLs, build a map from the source cards (data-citnumb to data-url) and from every chip that covers a single number. That map resolved 234 of the 242 chips in our samples. The other 8, six in local-restaurants and two in recipe, include a number that no single link on the page spells out.

“The Infatuation +2” is one of them: it covers numbers 6, 7 and 8, and number 7 never appears alone. cloro’s API returned all three pages for that chip (The Infatuation, Secret Chicago and Tripadvisor).

def resolve_chips(data):
    by_num = {card["num"]: card["url"] for card in data["cards"]}
    for chip in data["chips"]:
        if len(chip["sups"]) == 1:
            by_num.setdefault(chip["sups"][0], chip["href"])
    for chip in data["chips"]:
        chip["urls"] = [by_num.get(n) for n in chip["sups"]]  # None: not on the page
    return data


async def scrape(browser, prompt):
    page = await browser.new_page()
    try:
        await page.goto(COPILOT_SEARCH + quote_plus(prompt), timeout=30_000)
        iframe = await page.wait_for_selector(FRAME, timeout=30_000)
        frame = await iframe.content_frame()
        await wait_until_stable(frame)
        return resolve_chips(await frame.evaluate(EXTRACT_JS))
    finally:
        await page.close()


async def main(prompts, max_parallel=3):
    gate = asyncio.Semaphore(max_parallel)
    async with async_playwright() as p:
        browser = await p.chromium.launch()

        async def one(prompt):
            async with gate:
                return await scrape(browser, prompt)

        results = await asyncio.gather(*(one(q) for q in prompts), return_exceptions=True)
        await browser.close()
        return results


if __name__ == "__main__":
    print(asyncio.run(main(["best italian restaurants in chicago"])))

Run three at a time, not twelve

Each answer takes about 20 seconds to write, so when you scrape Bing Copilot in parallel, open browsers pile up fast. On 2026-11-07 we sent all 12 sample prompts to our own fleet at once, and 11 of them failed after 171 seconds. When we sent the same 11 three at a time, every one succeeded, in 19 to 43 seconds each.

That is what we saw on our infrastructure on one day, and we cannot state a Bing limit from it. Start your own scraper at three concurrent pages, as max_parallel does above, and raise it only while you watch the failure rate.

Country targeting comes from the exit IP

When you scrape Bing Copilot yourself, the location comes from where the browser connects. Our US request for “emergency plumber near me” came back about East Haven, Connecticut, and 9 of its 10 sources were pages about that town. For each country you want, you need a proxy exit in that country. For state-level local answers, you need exits in each state.

What breaks when you scrape Bing Copilot in production

• The class names are short and readable (b_ts_*, gs_*). That suggests people write them by hand and no build step generates them, so they should last longer than hashed names. Avoid the parts that are clearly generated: the indexed id #b_gsc_0, the insertion-id attributes (a new UUID on every render) and the data-bm counters.
• Matching on text is fragile. The “+3 more” button’s aria-label reads “3 more page sources are avaiable”, with Bing’s spelling. The follow-up heading appeared as “Follow-up question”, “Quick Follow-Up Question” and “One helpful follow-up” in our samples. Select by structure.
• Without a done flag, a stability check can stop early if the stream pauses. Log answers that end mid-sentence and widen the quiet window if you see them.
• 8 of 242 chips in our samples had a citation number with no URL on the page. Store None for those numbers and do not guess.
• Local answers can cite bing.com search pages and local listings. Filter or keep them on purpose.
• If you scrape Bing Copilot through a pool of mixed exit IPs, “near me” prompts come back about a mix of towns, which makes results hard to compare day to day.

The API path: one POST to cloro

If you do not want to scrape Bing Copilot yourself, the API does the same work: you send a prompt and a country, cloro opens Copilot Search, waits for the answer and parses it, and you get JSON back.

import requests

resp = requests.post(
    "https://api.cloro.dev/v1/monitor/bing/aimode",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "prompt": "best running shoes for flat feet under $150",
        "country": "US",
        "include": {"markdown": True, "html": True},
    },
    timeout=300,
)
result = resp.json()["result"]

print(result["text"][:200])
for source in result["sources"]:
    print(source["position"], source["url"])
for pill in result["citationPills"]:
    print(pill["citationPillId"], pill["domain"], pill["position"])

This is the response for that prompt from our samples, trimmed to one entry per list. The full answer had 9 sources and 23 citation pills.

{
  "success": true,
  "result": {
    "text": "The ASICS Gel Pulse 15, Brooks Glycerin 21, and Saucony Guide 19 are the strongest under-$150 choices for flat feet, offering wide bases, stability features, and cushioning suited to flat arches. ...",
    "sources": [
      {
        "position": 1,
        "label": "Best Running Shoes Under $150 in 2026: 8 Picks Worth the Money",
        "url": "https://runninggearlab.com/best-running-shoes-under-150/",
        "description": "The best running shoes under $150 in 2026, including Saucony Ride 19, Puma Velocity Nitro 4, Brooks Ghost 18, New Balance 880v15 and more."
      }
    ],
    "citationPills": [
      {
        "citationPillId": 1,
        "position": 3,
        "label": "Best Running Shoes for Flat Feet (2026): Tested Picks",
        "url": "https://www.alastairrunning.com/best-running-shoes-for-flat-feet/",
        "domain": "alastairrunning.com"
      }
    ],
    "markdown": "The **ASICS Gel Pulse 15**, **Brooks Glycerin 21**, and **Saucony Guide 19** are the strongest under-$150 choices ...",
    "html": "https://cdn.cloro.dev/results/4fbaeeae-0453-4283-8bb7-528f70d81c6d/page-1.html"
  }
}

sources lists the pages Bing read, in the order of the “Reading sources” list, then any other page a chip links to. citationPills has one entry per chip and source pair, and its position points back into sources. Headings, tables and the follow-up question are only in markdown, and html is a link to the rendered page.

On 2026-11-08 we timed 10 prompts through the sync endpoint, US, two at a time. All 10 returned an answer, in 15.0 to 26.5 seconds (median 19.4). Each answer had 9 to 11 sources and 14 to 39 citation pills.

A Bing AI Mode request costs 4 credits on the async API, the same as Google AI Mode. A sync request adds 2 credits, and a US state adds 2 more. On the Hobby plan ($100 a month for 250,000 credits), 4 credits is $1.60 per 1,000 async requests.

The free plan includes 500 credits a month. For batches, send the work to POST /v1/async/task or /v1/async/task/batch with taskType: "BING_AIMODE", and collect results by webhook or polling. See pricing for the other plans.

cloro is not the only API for this page. SerpApi and SearchApi both read bing.com/copilotsearch with engine=bing_copilot.

Which to choose

Scrape Bing Copilot yourself if you need a few hundred answers from one country, you already run browsers and proxies, and you can accept a few chips without URLs. The code above is enough to start.

Use an API if you need many countries or US states, a steady daily volume, or complete chip-to-source pairs for citation reporting. Our Bing AI Mode endpoint does the waiting, the parsing and the location for 4 credits a request. Our comparison of Bing Copilot APIs sets it beside SerpApi, SearchApi and the other vendors.