Google has officially announced the launch of the Multimodal Knowledge Agent (MKA) benchmark, designed to evaluate agents' reasoning capabilities when handling visually complex documents to address real-world knowledge work tasks.
Design Philosophy of the MKA Benchmark
The MKA benchmark was developed in collaboration with Artificial Analysis, primarily to assess agents' performance in solving knowledge work tasks. Most knowledge work benchmarks require only text resolution and fail to reflect the visual demands present in professional work. Google stated, “We built MKA to measure how well agents can handle this visually complex work.” Google
Practical Applications and Testing of the Benchmark
To ensure the benchmark aligns with real-world workflows, Google collaborated with domain experts from 10 industries and 83 professions to create a set of 200 task assessments for the MKA. Each task is a multi-step project that seasoned professionals take three to eight hours to complete. Google noted, “Each task reflects a multi-step project that takes a seasoned professional three to eight hours to complete…” Google
Performance of Gemini 4 Carbon
According to test results, Gemini 4 Carbon ranked second in quality among all tested models, with an average task completion cost six times lower than the highest-ranked model. Google stated, “Gemini 4 Carbon is on the pareto frontier of quality and price for the MKA eval.” Google
Users can now explore the benchmark through the Gemini API in Google AI Studio and evaluate their own models on the Artificial Analysis platform. Google has also released a set of 10 public tasks for anyone to check the format and test their agents.
Source: Google Official Announcement