ChatGPT has been out for 3 weeks. I sampled GitHub events for this month and 4 of the 6 most starred repositories in my sample are about it. One is a list of prompts, the others are a browser extension, an unofficial API client and a WeChat bot. Around me people say that this year everybody started an AI project. I said it too, a few times. I never checked it, but the data to check it is public. GH Archive records every public event on GitHub, hour by hour, since February 2011.
So the question was simple. Did 2022 have more new AI repositories than 2021, as a share of all new ones?
How I counted
Every hour of GH Archive is one gzipped file of JSON events, 30 to 200 MB in 2022. When somebody creates a repository there is a CreateEvent with ref_type set to repository. Its payload carries the description as it was at the moment of creation. That gives me 2 fields to read, the name and the description. Forks are a separate event and they are not in this count.
I took 2 hours of every day, 04:00 and 16:00 UTC, from 1 January 2021 to 24 December 2022. The first hour is daytime in Asia, the second one is morning in America. That is 1446 hours. 19 of them do not exist in GH Archive at all, 2 days in August 2021 and a week in October 2021. 1427 hours are left. My disk was almost full, so a small Python script read every file as a stream through curl and gzip and kept only the counts.
A repository counts as AI if its name or description says so. The words are machine learning, deep learning, neural, NLP, computer vision, TensorFlow, PyTorch, YOLO, “AI” as a separate word and a few more like Keras, BERT or Kaggle. Inside this I made a smaller group for the generative things: GPT, ChatGPT, OpenAI except its Gym, Stable Diffusion with its UIs like AUTOMATIC1111, DreamBooth, DALL-E, Midjourney, Whisper. Whisper is speech recognition, but it came from OpenAI in the same autumn and I kept it there.
Then I read 100 random repositories from 2022 by eye. 60 of them the label took and 40 a wider filter caught and the label dropped. The label is rarely wrong, but it misses a lot. “House price prediction” or a Vision Transformer project is usually machine learning and does not say the words. So every share below is a lower bound, measured the same way in both years.
October 2022
The first result was 1.05% of new repositories in 2021 and 0.91% in 2022. Lower. In October 2022 it fell to 0.65% and I already had a sentence ready about everybody leaving AI for something else.
Then I looked at October itself. In September my sample had 444,234 new repositories and in October 677,185. I opened one hour, 10 October 16:00 UTC. It had 15,267 new repositories and 12,856 of them had no description. The 8 most active accounts made 125 to 241 repositories each in that one hour, with random names. 2 of them are called pgslotzaa and PGSLOTTH888, which looks like Thai online slot sites. It was spam. And not only in that hour, most of October looks like this.
In 2021 the sample has 1,434,695 new repositories with a description and in 2022 1,427,656. Almost the same number. The 25% growth of GitHub in my sample came only from repositories without a description, their share went from 61.5% to 69.3%. So I took the share among repositories that have a description. My assumption is that spam accounts mostly do not write one. The October hour supports it.
With this denominator it is 1.36% in 2021 and 1.44% in 2022. Over the 24 months it jumps between 0.81% and 1.95%. December 2022 is 1.70%, but May 2021 was higher. If 2022 was an explosion of AI repositories, I cannot see it in what people create.
I was not the first to look. OSS Insight published a report in November 2022, with AI repositories found by their topics, for January to September. They wrote that the fast growth of open source AI was in 2016 and 2017 and after that it was slow. I have only 2 years, so I cannot say anything about 2016, but I also see no jump in 2022. Their period ends 5 weeks after Stable Diffusion released its weights and before ChatGPT.
Where the change is
The generative group is small, but it moves. In my sample it is 255 repositories in the whole 2021 and 1152 in 2022. 70% of the 2022 ones are from September to December.
In July and the first half of August it was 7 to 15 per week in my 14 sampled hours. Stable Diffusion released its weights on 22 August and from that week it was 18 to 34. ChatGPT came out on 30 November. That week has 66 and the next 3 weeks have 155, 130 and 126. The last of them is only 12 hours, because my data ends on 24 December. In December the generative group is 460 of 3268 AI repositories, 14%. As a share of everything created on GitHub it is still 0.15%.
A star is a WatchEvent in GH Archive and it has no description. So here I labeled repositories by the full name with the owner. This is how openai/whisper gets in.
The ordinary machine learning repositories got 2.64% of all stars in the sample in 2021 and 2.32% in 2022. No growth at all. The generative ones got 0.10% in 2021, 1.43% in September 2022 and 3.07% in December. In September the most starred generative repositories were CompVis/stable-diffusion and openai/whisper. The code of ChatGPT is not on GitHub. In December the first one is f/awesome-chatgpt-prompts, a list of text prompts.
In December 699 generative repositories got stars in my sample. Together they have 3.07% of all stars and the first 3 of them have 22% of that.
What I did not check
- Topics. OSS Insight used them and I used names and descriptions. A description added after creation is not visible to me.
- 22 hours of every day. My numbers are counts in a sample. The real counts are much bigger and I did not try to scale them up.
- Stars of repositories that only the owner name marks as AI. I kept the full list only for names that passed the wider filter, plus the top 300 of every hour.
- Private repositories. GH Archive sees only public events.
- If the 1.4% is the same people. A student who makes a new
ML-Homework-3every week counts every week.
December is not finished. If the ChatGPT weeks continue in January, the share among new repositories will finally move. We’ll see.