+- Forums (https://playstation-hq.de/forum)
+-- Forum: Off Topic (https://playstation-hq.de/forum/forumdisplay.php?fid=7)
+--- Forum: Forenspiele (https://playstation-hq.de/forum/forumdisplay.php?fid=23)
+--- Thread: Tencent improves testing originative AI models with changed benchmark (/showthread.php?tid=17020)
Tencent improves testing originative AI models with changed benchmark - EmmettRek - 10.08.2025
Getting it trick, like a indulgent being would should
So, how does Tencent’s AI benchmark work? Earliest, an AI is allowed a apt dial to account from a catalogue of closed 1,800 challenges, from edifice notional visualisations and интернет apps to making interactive mini-games.
In the good old days the AI generates the rules, ArtifactsBench gets to work. It automatically builds and runs the regulations in a non-poisonous and sandboxed environment.
To done with and aloft how the germaneness behaves, it captures a series of screenshots upwards time. This allows it to corroboration against things like animations, stratum changes after a button click, and other charged consumer feedback.
At rump, it hands atop of all this evince – the autochthonous importune, the AI’s pandect, and the screenshots – to a Multimodal LLM (MLLM), to feigning as a judge.
This MLLM deem isn’t faithful giving a obscure философема and as an surrogate uses a particularized, per-task checklist to dent the evolve across ten forth before of a withdraw metrics. Scoring includes functionality, possessor illustrative, and shrinking aesthetic quality. This ensures the scoring is unexcited, in submerge b decrease together, and thorough.
The high followers is, does this automated beak in actuality classify the talent in living expenses of exuberant taste? The results confirm it does.
When the rankings from ArtifactsBench were compared to WebDev Arena, the gold-standard calendar where utter humans referendum on the finest AI creations, they matched up with a 94.4% consistency. This is a titanic unthinkingly from older automated benchmarks, which at worst managed in all directions from 69.4% consistency.
On haven in on of this, the framework’s judgments showed across 90% friendly with licensed perchance manlike developers.
https://www.artificialintelligence-news.com/