Traffic Comparison between OpenAI and Claude
published at: 2024-09-29
OpenAI recently released their o1 model in preview version, as the final model is still under training. This model introduces a new paradigm where it is taught during training to do something called chain-of-thoughts. With this, the model can "reason" before providing an answer, which improves its performance.
Before the release of o1, the Claude Sonnet 3.5 from Anthropic was the best model for handling more complex tasks and codes.
But with the release of o1, it proved very effective for the tasks I previously used Sonnet 3.5 for.
Its ability to perform chain-of-thoughts on the fly (I'll explore this topic more in the future) significantly improved its capability.
Compared to Sonnet 3.5, there's a noticeable improvement.
However, the main point of this article isn't to compare the two models—there are several articles on the internet about that—but to find out if, after the release of o1, the number of people using Sonnet 3.5 has decreased.
Google Trends measures the number of Google searches for specific terms. I searched for OpenAI o1 and Claude Sonnet 3.5. Since o1 was launched on 2024-09-21, I only considered the last seven days (2024-09-21 to 2024-09-28) to check the post-launch period.
Note
Using Google Trends is useful to see how people are searching for content organically. However, it's important to remember that it searches by keywords, so it is possible that there are other keywords where Sonnet is more searched than the one I used here, and vice versa.
Right after the release, searches for o1 were higher than for Sonnet, but throughout the week they converged and are now at the same level.
This behavior is interesting because I thought o1 would remain more searched than Sonnet for a longer time.
To try to make a fairer analysis, I retrieved some data from SimilarWeb. This website monitors web traffic through proprietary solutions and models. In the end, it estimates who actually accesses the site, while Google Trends shows who searched, but not necessarily who converted—meaning who clicked through to the site.
Looking at the total number of visits after the launch of Sonnet 3.5, visits to the Claude website surged and remain at a level over 100% growth compared to the previous period.
The opposite behavior can be seen on the OpenAI website. After the release of Sonnet 3.5, visits dropped until the end of August. However, in September, visits grew compared to June. It’s important to note that the growth rate between the two sites isn’t comparable because OpenAI's website has a much larger number of visits, so the growth tends to be smaller.
Zooming in on the last two weeks of September, it shows a drop after the launch of o1. This is quite strange, but we only have data from the 21st to the 24th, and the 21st and 22nd are weekends—days when people typically access websites less. Thus, the numbers might be underestimating the real impact of the launch on visit counts.
Despite OpenAI's o1 release being impactful in benchmarks, it hasn't yet translated into more website visits.
One possible reason for this could be the fact that this model is only available to paid ChatGPT users, unlike Sonnet 3.5, which is available for free and paid users through Claude's chat interface.
Moreover, it's still an analysis with a small post-launch sample, but we should see a clearer trend in the coming weeks.
One hypothesis is that this release will accelerate the addition of new paid users to the ChatGPT service, which will then be reflected in increased traffic.