According to data from research firm Epoch AI, the cost of training AI models has exploded in the past year alone. This development clearly shows how much more complex and powerful AI models have become in a short period of time. In March 2023, OpenAI released the latest ChatGPT version, which sparked the global AI hype. Google followed suit in December with its advanced AI model Gemini.

Both systems were significantly more expensive to train than previous AI models, and their development may have cost hundreds of millions of dollars, according to Epoch AI’s press release. The cost of training Gemini, a large language model that can be input using text, voice commands, and images, reportedly ranged from $30 million to $191 million, even without considering employee salaries. According to Epoch AI, these can account for 29 to 49 percent of the final price. ChatGPT-4, the latest edition, had engineering creation costs of $41 million to $78 million, according to the source. OpenAI CEO Sam Altman has said in the past that the model cost more than $100 million, which confirms the calculations.

Looking back, the cost of previous AI models was much lower. ChatGPT-3 cost only about $2-4 million in 2020, while training Gemini’s predecessor PaLM cost between $3-12 million in 2022, considering computational costs alone. Even at these prices, it might have been difficult for academic or other public institutions traditionally active in AI research to keep up with the latest AI development.

This is essentially impossible given the 2023 cost estimates, as Epoch AI notes, mentioning as a possible solution the National AI Research Resource created by the Biden administration in late 2023. It would provide researchers and students with access to relevant AI tools and award grants, but it is still in the pilot phase. The executive order that created the resource focuses primarily on setting standards for AI safety and privacy, such as strengthening consumer rights vis-à-vis algorithms as well as workers’ rights in the face of changes in the workplace.

AI for consumption?

While ChatGPT-4 was updated to support voice and images in fall 2023, as its name suggests, it was initially based on centralized text input, while Gemini and its app were designed from the start as a multimodal LLM. This explains why ChatGPT’s initial training costs may have been lower. On the other hand, Gemini’s general focus on app deployment, such as asking users to take pictures with their smartphones, select features in them, and have them analyzed, could have justified higher costs.

Gemini also includes e-commerce-related features, such as showing where to buy something that appears in an image, similar to a Google search (Shopping). This shows how Google is applying its brand identity as a search engine to AI models, while AI-focused company OpenAI had to build its identity and strengths in the AI ​​space from scratch. It also raises the question of whether AI in the future will tend more towards commercial support functions, as expected by the Biden administration, rather than original text generation, ChatGPT’s most publicized feature.

OpenAI’s text-to-image model DALL-E was significantly less expensive in 2021 than the LLMs built around that time, including ChatGPT’s version 3 from 2020. According to Epoch AI, it only cost between $118,000 and $335,000 to produce. DALL-E 3, the latest successor version, is now part of the enhanced ChatGPT versions for paying customers. The price of owning hardware is always lower in this calculation than in the cloud computing approach, because it uses amortized costs, i.e. only takes into account the proportion of the total lifetime cost of a hardware component relative to the time spent training the respective AI model.

—

Mapped by Statista