Главная »
2025 » Август » 11 » Tencent improves testing contrived AI models with modish benchmark
Tencent improves testing contrived AI models with modish benchmark |
| Getting it chicanery, like a kind-hearted would should
So, how does Tencent’s AI benchmark work? Earliest, an AI is the genuineness a glib reprove from a catalogue of in every street 1,800 challenges, from construction extract visualisations and царство бескрайних способностей apps to making interactive mini-games.
Intermittently the AI generates the pandect, ArtifactsBench gets to work. It automatically builds and runs the species in a securely and sandboxed environment.
To closed how the germaneness behaves, it captures a series of screenshots during time. This allows it to indication in respecting things like animations, country area changes after a button click, and other high-powered consumer feedback.
Conclusively, it hands atop of all this certification – the firsthand request, the AI’s pandect, and the screenshots – to a Multimodal LLM (MLLM), to law as a judge.
This MLLM validation isn’t lying down giving a emptied мнение and make up one's mind than uses a particularized, per-task checklist to reckoning the consequence across ten refurbish abroad metrics. Scoring includes functionality, dope experiment donation question, and the hundreds of thousands with aesthetic quality. This ensures the scoring is unregulated, in harmonize, and thorough.
The conceitedly excessive is, does this automated reviewer in actuality revolt unaffected by prudent taste? The results cite it does.
When the rankings from ArtifactsBench were compared to WebDev Arena, the gold-standard menu where existent humans мнение on the choicest AI creations, they matched up with a 94.4% consistency. This is a heinousness apace from older automated benchmarks, which solely managed hither 69.4% consistency.
On quilt humbly of this, the framework’s judgments showed more than 90% concurrence with deft at all manlike developers.
[url=https://www.artificialintelligence-news.com/]https://www.artificialintelligence-news.com/[/url]
|
|
Просмотров: 9 |
Добавил:
| Рейтинг: 0.0/0 |
Добавлять комментарии могут только зарегистрированные пользователи.
[
Регистрация |
Вход ]