E1 · IndexNow and Bing
25 minCheck the sitemap in Bing Webmaster, publish the IndexNow key, and notify 3 new or changed URLs that return 200.
Done when: Bing shows the sitemap with a success status and the IndexNow notification returned 200 or 202.
Lesson 5 · 18 min
Bing matters twice; the AI bots show up in the server log
Bing feeds Copilot and has the only official citations dashboard; and one study found a good share of ChatGPT's citations among the top Bing results. Telling Bing costs little. On the other side, the server log tells you who really reads the site: which AI bots, for what purpose, and which ones are fake.
Start the lesson
In this lesson you will
- 1The two reasons to care about Bing (and what is a study, not a fact)
- 2Notify IndexNow only with URLs that return 200
- 3Read the log by purpose: search, user action, training
- 4Spot fake bots and CDN blocks in the log
Goal: check the site in Bing with IndexNow and read in the log which AI bots really visit, separated by purpose.
01Who finds you, who reads you, who cites you
Click each block.
Click each block in the diagram to see what it does.
Notify Bing, then check the log to see who really came.
02In practice, in four steps
Tap each step.
1. Sitemap in Bing
Check in Bing Webmaster that the sitemap was read successfully (course 1, lesson 3). Without this, neither AI Performance nor IndexNow pays off.
2. IndexNow
Publish the key in a .txt file at the site root and notify the new URLs after each publication. The method's script checks for the 200 before notifying.
At INEMA: 109 URLs notified with a 202 response on 09/24/2026 and 90 URLs with 200 on 10/02/2026.
3. Get the log
On the server (nginx or Apache), copy the access log to your computer. Behind a CDN, use the CDN's log: the server's log only sees what got past it.
A person's IP is personal data (LGPD): before showing it to anyone, keep only the bot lines.
4. Read by purpose
Run botlog (inemaseo-sistemas, MIT) or filter by hand on User-Agent. Look at three things: which purpose shows up, how many fakes there are, and whether any search bot got a 403.
Common reading
Only training bots in the log and no search bots: the site may be outside the index of the search AIs. Check robots.txt and the CDN (course 1, lesson 6).
03IndexNow and botlog
The first block is from the inemaSEO method (model only: it needs your key). The second reads the log on your computer; detailed teaching in course 7.
avisar_buscadores.sh (inemaseo-metodo)
# urls.txt: one URL per line, all on the same host # the IndexNow key is published at https://seusite.com/<chave>.txt INDEXNOW_KEY=<chave> tools/avisar_buscadores.sh urls.txt # checks that each URL returns 200 before notifying; stops if any does not return 200
botlog (inemaseo-sistemas)
git clone https://github.com/inematds/inemaseo-sistemas && cd inemaseo-sistemas/ferramentas/botlog # copy the server log (read-only) ssh usuario@SEU_SERVIDOR 'cd /var/log/nginx; zcat -f access.log.*.gz; cat access.log.1 access.log' > /tmp/site.log # report by purpose, with the IP checked against the official lists python3 botlog.py analisar /tmp/site.log --sitemap https://seusite.com/sitemap.xml --saida saida/site # LGPD: before sharing, keep only the bot lines with an official IP python3 botlog.py filtrar /tmp/site.log --saida /tmp/so-robos.log
Example from the repository
botlog on the real log of desenho.inema.pro (10/05 to 10/19/2026), already filtered to only bots with an official IP:
8 linhas, 8 de robô (treino 6, busca 2); falsos 0 robo finalidade acessos paginas robots_txt 4xx/5xx GPTBot treino 4 2 0 0 ClaudeBot treino 2 2 1 0 OAI-SearchBot busca 2 1 2 1
In the repository's assembled sample (documentation IPs, official User-Agents), the report flags:
- 4 fakes: GPTBot, Googlebot and PerplexityBot coming from an IP outside the official list.
- A 403 for ClaudeBot, which simulates a CDN block.
- A sitemap page that no bot read.
- The log shows a request, not a citation.
- Behind a CDN, the IP may be the CDN's: use its log.
- An old IP list creates false "fakes": update it.
04Practice: notify and check
No access to the log? Do E2 with the example from the repository.
E2 · Bots by purpose
30 minRun botlog on your site's log (or on the REAL example in the repository) and note the requests by purpose, the fakes, and the 403s for AI bots.
Done when: a note with the count by purpose (search, user action, training), the number of fakes, and the list of 403s (or "none").
05Test what you learned
The explanation appears after your answer.
Question 1 of 4
"ChatGPT uses the Bing index." How do you say this honestly?skill
Correct
A hint from a study, with a small sample. A good reason to notify Bing, not a promise about ChatGPT.Not this one
A hint from a study, with a small sample. A good reason to notify Bing, not a promise about ChatGPT.Review: Who finds you, who reads you, who cites youQuestion 2 of 4
The log shows 30 requests from "GPTBot" coming from an IP outside OpenAI's list. What are they?
Correct
Anyone can write a User-Agent. Without the IP on the official list, do not count it as an OpenAI bot.Not this one
Anyone can write a User-Agent. Without the IP on the official list, do not count it as an OpenAI bot.Review: Who finds you, who reads you, who cites youQuestion 3 of 4
OAI-SearchBot gets a 403 in the log, but robots.txt allows it. Where do you look?
Correct
robots.txt asks; the CDN enforces. A 403 for an AI bot is usually a CDN rule.Not this one
robots.txt asks; the CDN enforces. A 403 for an AI bot is usually a CDN rule.Review: IndexNow and botlogQuestion 4 of 4
Why does searching for Google-Extended in the log not work?
Correct
Google-Extended controls use for training Gemini, with no bot of its own. It does not affect Google Search.Not this one
Google-Extended controls use for training Gemini, with no bot of its own. It does not affect Google Search.Review: Who finds you, who reads you, who cites youQuiz complete
You got {a} of {n} right. Get them all right and the lesson is marked as complete.Choose an answer. The explanation appears after you try.
06Takeaway
BingNotify with IndexNow only URLs that return 200; Bing feeds Copilot and the only citations dashboard.
LogRead by purpose, check the IP, and look for 403s on AI bots.
2026 yardstickChatGPT–Bing is a hint from a Seer study, not a statement from OpenAI. A log shows requests, not citations.