OpenAI unveils GPT-4o mini, its most affordable model, with an 82% score on MMLU and up to 60% lower pricing than GPT-3.5 Turbo.
Launched on July 18, 2024, GPT-4o mini is the most economical model in OpenAI's range, designed to democratize access to generative AI at scale.
Prices and technical characteristics
The model is priced at $0.15 per million tokens in input and $0.60 per million in output — up to 60% lower than GPT-3.5 Turbo, and out of all proportion to OpenAI's previous flagship models. Its context window reaches 128,000 tokens, with up to 16,000 output tokens per request, and a knowledge base as of October 2023. It currently supports text and vision via API; audio and video support is announced for later.
Competitive performance
OpenAI announces a score of 82% on the MMLU (Textual Intelligence and Reasoning) benchmark, ahead of Gemini Flash (77.9%) and Claude Haiku (73.8%). In mathematics (MGSM), GPT-4o mini reaches 87%, compared to 75.5% for Gemini Flash and 71.7% for Claude Haiku. In coding (HumanEval), it scores 87.2%, compared to 71.5% and 75.9% respectively for its two direct competitors. On multimodal reasoning (MMMU), it scores 59.4%, ahead of Gemini Flash (56.1%) and Claude Haiku (50.2%).
Targeted use cases
The model is particularly suitable for uses requiring multiple or parallel API calls, large context (entire codebase, conversation history) or real-time responses such as customer support chatbots. OpenAI says it has tested the model upstream with partners such as Ramp and Superhuman, which report better results than with GPT-3.5 Turbo on structured data extraction and email response generation.
Security
GPT-4o mini inherits the same guardrails as GPT-4o: filtering of sensitive training data, post-training alignment by RLHF, and evaluation by more than 70 external experts (psychology, misinformation) as part of OpenAI's Preparedness Framework.
Read the full article on the original source
Go to the source →