How AI-Ready Are Japan’s Largest Companies? We Checked 30 Corporate Websites
Reserved
·
November 9, 2026
·
6 min read
We checked how ready the websites of 30 of Japan's largest listed companies are for AI search. Of the 26 we could analyse, only 2 publish an llms.txt file, 9 describe their organisation in structured data, and 3 name any AI crawler in their robots.txt. Only one, Nintendo, blocks the major AI crawlers.
In other words, nearly all of these companies leave AI assistants free to read their sites, but give them little help understanding what they read. Most of the gaps take very little effort to close.
Key findings
- AI crawlers are almost always allowed. 25 of 26 sites let GPTBot, ClaudeBot and PerplexityBot in, mostly because they never mention them.
- Few companies have an AI policy in robots.txt. Only Recruit Holdings, Nintendo and Takeda name AI crawlers at all.
- 11 of 26 sites (42%) have no robots.txt file on the domain we checked.
- Structured data is patchy. 12 of 26 homepages include any schema.org data, and 9 identify the company as an Organization.
- llms.txt is rare. Takeda and Seven & i Holdings are the only two that publish one.
- Some homepages show crawlers almost nothing. Recruit Holdings and Nintendo serve very little text in their raw HTML, which crawlers that don't run JavaScript may struggle with.
- Four sites refused our automated requests, which suggests bot protection that may also affect AI crawlers.
Why AI readiness matters
People increasingly ask AI assistants about companies: what they make, who runs them and how to contact them. Assistants answer from what they can read. A site that is easy to crawl and states its facts clearly is more likely to be described accurately. No website can make an assistant recommend it, but a clear website makes it easier for assistants to describe you correctly.
We checked five things:
- robots.txt, the file that tells crawlers what they may access. We noted whether AI crawlers are named, allowed or blocked.
- llms.txt, a newer and unofficial convention: a plain-text summary of a site for AI systems.
- Structured data, the schema.org markup that states facts such as company name, address and logo in a machine-readable form.
- Raw HTML content, meaning how much text the homepage contains before any JavaScript runs.
- Basic metadata, including meta descriptions, social sharing images, links to language versions (hreflang) and sitemap references.
Results
| Check | Sites | Share |
|---|---|---|
| Allow major AI crawlers | 25 of 26 | 96% |
| Name any AI crawler in robots.txt | 3 of 26 | 12% |
| Have a robots.txt file | 15 of 26 | 58% |
| List a sitemap in robots.txt | 13 of 26 | 50% |
| Publish llms.txt | 2 of 26 | 8% |
| Any structured data on the homepage | 12 of 26 | 46% |
| Organization structured data | 9 of 26 | 35% |
| Meta description | 21 of 26 | 81% |
| Social sharing image (og:image) | 19 of 26 | 73% |
| Links to language versions (hreflang) | 13 of 26 | 50% |
Company by company
| Company | robots.txt | AI crawlers | llms.txt | Organization schema |
|---|---|---|---|---|
| Advantest | Yes | Allowed | No | Yes |
| Daiichi Sankyo | No | Allowed | No | Yes |
| Daikin Industries | Yes | Allowed | No | No |
| Fast Retailing | No | Allowed | No | No |
| Hitachi | Yes | Allowed | No | Yes |
| Honda Motor | No | Allowed | No | Yes |
| Hoya | Yes | Allowed | No | No |
| Itochu | No | Allowed | No | No |
| Japan Tobacco | Yes | Allowed | No | No |
| KDDI | No | Allowed | No | No |
| Keyence | Yes | Allowed | No | Yes |
| Mitsubishi Heavy Industries | Yes | Allowed | No | No |
| Mitsubishi UFJ Financial Group | No | Allowed | No | No |
| Mitsui & Co. | No | Allowed | No | No |
| Murata Manufacturing | Yes | Allowed | No | No |
| Nintendo | Yes | Blocked | No | No |
| NTT | Yes | Allowed | No | Yes |
| Recruit Holdings | Yes | Allowed by name | No | No |
| Seven & i Holdings | No | Allowed | Yes | Yes |
| Shin-Etsu Chemical | Yes | Allowed | No | No |
| SoftBank Group | Yes | Allowed | No | Yes |
| Sumitomo Mitsui Financial Group | Yes | Allowed | No | No |
| Takeda Pharmaceutical | Yes | Allowed | Yes | Yes |
| Tokio Marine Holdings | No | Allowed | No | No |
| Tokyo Electron | No | Allowed | No | No |
| Toyota Motor | No | Allowed | No | No |
"Allowed" means that robots.txt does not block GPTBot, ClaudeBot or PerplexityBot, including when there is no robots.txt at all. We could not analyse Sony Group, Mitsubishi Corporation, Mizuho Financial Group or Fujitsu, whose sites refused or timed out our requests.
What stood out
Nintendo is the only one to block AI crawlers
Nintendo's robots.txt blocks GPTBot, ClaudeBot, Google-Extended, CCBot, Applebot-Extended and Bytespider. For a company whose value rests on its characters and intellectual property, that is an understandable choice. It also shows that blocking is a policy decision, and one that most of these companies have not made either way.
Recruit Holdings has an explicit AI policy
Recruit's robots.txt names GPTBot, ClaudeBot, PerplexityBot and another AI crawler individually, and allows them. That is the clearest deliberate position in our sample. Its root homepage, however, is a short JavaScript redirect to a language-specific page. A crawler that does not run JavaScript sees almost no text there.
Takeda and Seven & i are early llms.txt adopters
Takeda's llms.txt lists its official country and language sites, which helps an AI system tell legitimate Takeda domains from imitations. That is useful for a pharmaceutical company. Seven & i's file is in Japanese and points to its investor relations pages.
Many companies have no robots.txt
Eleven sites returned no robots.txt on the domain we checked, including those of Toyota, MUFG, KDDI and Honda. Without the file, everything is allowed, so this does not block anyone. It does mean the companies are not pointing crawlers to a sitemap, and are not stating any policy on AI use of their content.
Bot protection may affect AI crawlers too
Four sites blocked or throttled our simple, clearly identified research requests. Their security systems may be treating AI crawlers the same way. If so, those companies may be less visible to AI assistants than they intend, without anyone having decided that.
How to make a corporate website AI-ready
- Decide your AI crawler policy. Allow or block GPTBot, ClaudeBot, PerplexityBot and Google-Extended deliberately, and write the decision into robots.txt.
- Check your bot protection. Make sure your firewall or CDN is not blocking the crawlers you meant to allow.
- Add Organization structured data with your official name in English and Japanese, logo, address, website and official social profiles.
- Serve key facts in the HTML. Your company description, products and contact details should not depend on JavaScript to appear.
- List your sitemap in robots.txt, and link your language versions with hreflang.
- Consider llms.txt. It costs little, but set your expectations: an Ahrefs study found that 97% of llms.txt files received no requests. Treat it as optional, and do the steps above first.
Method
We chose 30 well-known large-cap companies listed on the Tokyo Stock Exchange from across sectors. This is a selection, not an official index. On 9 November 2026, we requested each company's robots.txt, llms.txt and homepage once, from its main corporate domain, with a user agent identifying our research.
- robots.txt: we checked rules for ten AI crawlers, including GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, Google-Extended and CCBot. A crawler counts as blocked when its rules, or the general rules, disallow the whole site.
- llms.txt: we counted a file only if it returned plain text with real content, not an error page.
- Homepage: we read the HTML as served, without running JavaScript, and recorded structured data types, metadata and visible text.
Limitations: we checked one domain per company, and many run several. Results reflect a single day, and sites change. A site that blocked us may behave differently for real AI crawlers. We will be glad to correct the record: if you represent one of these companies and something has changed, contact us.
Frequently asked questions
Do Japanese companies block AI crawlers?
Rarely, in our sample. Only Nintendo blocks the major AI crawlers. 25 of the 26 sites we could analyse allow them, mostly by not mentioning them at all.
What is llms.txt?
It is a proposed plain-text file at a website's root that summarises the site for AI systems. It is not an official standard, and evidence that AI systems use it is limited.
Does structured data help with AI search?
Google says no special markup is required for its AI features. Structured data still states facts unambiguously, such as a company's official name and logo, which helps any system that reads it describe the company accurately.
Should a company block AI crawlers?
It depends on what you value more: protecting your content from use in AI training, or being described accurately when people ask AI assistants about you. The important thing is to make the decision deliberately.
About this research
Reserved is a technical partner based in Tokyo. We help companies in Japan with AI integration and websites. If you would like the same check run on your site, get in touch. You are welcome to cite this research with a link to this page.
