Under 1% of websites load a script that calls itself AI

ChatGPT came out on the last day of November and this month everybody around me talks about AI on the web. I had a feeling that it was already there for years, in third party scripts that every site pulls in. Chat widgets, recommendations, “smart” search. I never put a number behind this feeling. It came from vendor landing pages and I had never counted.

HTTP Archive crawls millions of sites every month and runs Wappalyzer on every page. So the number exists somewhere and I decided to find it.

How I counted

Wappalyzer is open source and keeps a catalog of technologies on GitHub, with a short description of each one. On 30 November 2022 it had 3470 technologies. I did not want to decide myself what is AI, so I let the vendors decide. A technology counts if its own description says “AI”, “artificial intelligence”, “machine learning”, “deep learning”, “neural”, “natural language”, “NLP” or “computer vision”. The label is the vendor’s marketing. Still, it is what the vendor tells you about the script you put on your page.

For each of these technologies the technology report of HTTP Archive gives the number of origins where it was detected, per month and per client. I took mobile, all ranks, January to November 2022. December is not published yet.

There is one limitation I could not get around with these numbers. The report counts each technology separately. A site with 2 AI scripts is counted twice, so the sum over technologies is an upper bound. The largest single technology is a lower bound. The real share is between them.

The first count

My first version was wide. Besides the AI words it also took “chatbot” and “conversational”, because a chat widget was the first thing I imagined. It also matched “.ai” in names.

It said 0.85% of mobile origins in January and 1.31% in November. Up by more than half in 10 months, in the year when everybody started to talk about AI. I almost wrote this down as the result.

Then I looked at what was inside. SendPulse is in it with 7,140 origins. It is an email marketing platform and its description lists Facebook and WhatsApp chatbots among its channels. Marchex got in as “call and conversational analytics”. Rise.ai and Kontent.ai got in by the domain in their name, one is a loyalty tool and the other is a CMS. I removed the chatbot words and the “.ai” match and kept only what the description itself claims.

With the strict label it is 46 technologies in November and 76,848 origins together. Dynatrace is in the catalog twice, as Dynatrace and Dynatrace RUM, so I merged them into one vendor and got 67,340. That is 0.72% of 9,344,044 mobile origins. The largest single vendor, Ezoic, alone is 20,573, which is 0.22%. So the real share is between 0.22% and 0.72%.

Who these vendors are

I expected chat bots on top. They are not. The top of the list in November:

Ezoic and Dynatrace together make 47% of the merged sum. The first is an optimisation and ads platform for publishers, the second is monitoring. If someone tells me that the web depends on AI scripts, in 2022 almost half of that dependency is 2 companies. Neither of them is what I pictured.

None of the top descriptions talks about a model in the browser. I also looked for TensorFlow.js and ONNX in the catalog, there is no entry for them at all.

The growth was the catalog

The wide count grew by more than half and the strict count also grew, from 0.58% in January to 0.72% in November. I wanted to know where this growth came from.

The catalog itself changes during the year. In January 28 of the strict technologies had at least one origin, in November 46. I took only the 28 technologies that were already detected in January and followed them through the year.

Their share went from 0.58% to 0.54%. In absolute numbers they grew from 37,131 to 50,360 origins. But the crawl grew too, from 6.35 million mobile origins in January to 9.34 million in November, with the big jump in August. So the vendors that were already known did not grow faster than the crawl.

All of the growth in my strict number comes from the 18 technologies that were not there in January. I opened the catalog as it was on the last day of 2021. 16 of the 18 had no entry at all, so for them it is new rules. A new rule probably finds sites that had the script before, but I did not check this. Cxense had a rule, but the January data has no Cxense. It appears in February with 4,457 origins. That is a gap in the data and not a new rule. Onfido also had a rule and has only 31 origins.

The wide count does the same: its January set went from 0.85% to 0.79%.

Bigger sites

Among the top 10,000 sites the picture is different. There are 8,631 mobile origins in this bucket and the strict sum is 657. That is 7.6% as an upper bound, about 10 times the share of the whole web. Cxense leads with 179, then Dynatrace with 133. Ezoic is not in the top 5. Scripts that call themselves chatbots are only 97 there, 1.1%.

What I did not check

The real union. With per technology counts I only have bounds. HTTP Archive has the page level data in BigQuery. A query over it would give the exact number of sites with at least one such script and I did not run it.

Whether the label means anything. “Powered by machine learning” in a vendor description is a claim and I did not check any vendor’s model. The count also misses vendors that use machine learning and never say so.

Desktop. The strict sum is higher there, 1.17% before merging Dynatrace. I did not look into why. I took mobile because it is the bigger crawl.

Models in the page. TensorFlow.js and similar libraries are not in the catalog, so I cannot see them here.