Home · Technology · Sep 19 archive
Vals Aims to Set New AI Benchmarking Standards
Confirmed
In Short: Vals aims to fix this imperfect system by setting new standards for measuring AI capabilities. The company's efforts come as the industry continues to advance, making it crucial to have accurate and up-to-date benchmarking methods.
Vals, a startup founded in 2024, is working to address the shortcomings of current AI benchmarking systems, Krishnan, the founder, told TechCrunch. The company, which started the year with only eight employees, has already expanded to a team of 25.
Benchmarking has become a standard practice for AI companies to validate their models' capabilities and distinguish themselves from competitors. However, companies have found ways to outmaneuver older benchmarking systems, which are not designed to measure the capabilities of modern AI models.
In a separate development, the Indian government launched the Elderly Friendly Office Certification (EFOC) program, which aims to address common issues faced by senior citizens when visiting offices. The program will inspect offices against clear standards and award them Gold, Silver, or Bronze certificates.
The EFOC program draws inspiration from models in the UAE, the World Health Organization’s Age-Friendly Cities network, and the Certified Age-Friendly Employer programme in the United States. It is the first time such standards have been implemented in India.
Meanwhile, the Genesis Mission, a whole-of-government initiative led by the White House Office of Science and Technology Policy, aims to double the productivity and impact of American science and engineering within a decade by leveraging AI-enabled scientific discovery and innovation.
What this adds
Vals' rapid growth from eight to 25 employees underscores the demand for improved benchmarking standards in the AI industry.
What's confirmed
- Vals, a startup founded in 2024, is working to address the shortcomings of current AI benchmarking systems, Krishnan, the founder, told TechCrunch. The company, which started the year with only eight employees, has already expanded to a team of 25.
- Benchmarking has become a standard practice for AI companies to validate their models' capabilities and distinguish themselves from competitors. However, companies have found ways to outmaneuver older benchmarking systems, which are not designed to measure the capabilities of modern AI models.
- In a separate development, the Indian government launched the Elderly Friendly Office Certification (EFOC) program, which aims to address common issues faced by senior citizens when visiting offices. The program will inspect offices against clear standards and award them Gold, Silver, or Bronze certificates.
- The EFOC program draws inspiration from models in the UAE, the World Health Organization’s Age-Friendly Cities network, and the Certified Age-Friendly Employer programme in the United States. It is the first time such standards have been implemented in India.
- Meanwhile, the Genesis Mission, a whole-of-government initiative led by the White House Office of Science and Technology Policy, aims to double the productivity and impact of American science and engineering within a decade by leveraging AI-enabled scientific discovery and innovation.
What's still developing
- 25% off tickets now Back by popular demand: Save up to $300 on Disrupt Benchmarking has become the industry norm for how AI companies validate their models’ capabilities and, when the metrics swing in their favor, stand out from competitors and advertise their superiority.
- Vals, a startup formed in 2024, says that it is on a mission to fix this very imperfect system.
- The move aims to address common problems faced by senior citizens when they visit offices, such as long queues, stairs without handrails, forms in small print, and counters that require climbing stairs, etc.
- “Our contribution to the Genesis Mission exemplifies NIST’s ongoing commitment to advancing U.S. leadership in critical technologies that are vital to America’s economic and national security,” said Under Secretary of Commerce for Standards and Technology and NIST Director Arvind Raman.