Home · World · Sep 19 archive
Vals Aims to Set AI Benchmarking Standards
Confirmed
In Short: Vals aims to address these issues by setting new standards for AI benchmarking. The company's mission is to create a more accurate and reliable system for evaluating AI models.
Vals, a startup founded in 2024, is working to improve the benchmarking of AI models, Krishnan, the founder, told TechCrunch. The company has grown rapidly, tripling its team from eight to 25 people.
Benchmarking has become the industry norm for validating AI models' capabilities and distinguishing them from competitors. However, companies have also found ways to outsmart legacy benchmarking systems, which are often outdated and not designed to measure modern AI capabilities.
In a separate development, the Indian government launched the Elderly Friendly Office Certification (EFOC) program, which sets technical standards for offices to better accommodate senior citizens. The program includes clear, common standards for inspections and awards offices Gold, Silver, or Bronze certificates.
The EFOC program addresses common issues faced by senior citizens, such as long queues, stairs without handrails, and forms in small print. This is the first time such standards have been implemented in India.
Meanwhile, the U.S. government is focusing on advancing AI through the Genesis Mission, a whole-of-government initiative led by the White House Office of Science and Technology Policy. The mission aims to double the productivity and impact of American science and engineering within a decade.
President Trump, in a social media post, announced new fuel economy standards that roll back Biden-era vehicle rules. The changes are expected to lower car prices but may also increase fuel consumption.
The Department of Transportation estimates that the new standards will reduce the typical price of a new car by about $930 but could lead to an increase in fuel consumption by about 100 billion gallons through 2050.
What this adds
Vals' rapid growth and mission to improve AI benchmarking standards reflect the evolving needs of the industry.
The EFOC program in India highlights the importance of setting technical standards to improve accessibility for senior citizens.
The Genesis Mission underscores the U.S. government's commitment to advancing AI and scientific innovation.
The rollback of fuel economy standards by the Trump administration may have significant economic and environmental implications.
What's confirmed
- Vals, a startup founded in 2024, is working to improve the benchmarking of AI models, Krishnan, the founder, told TechCrunch. The company has grown rapidly, tripling its team from eight to 25 people.
- Benchmarking has become the industry norm for validating AI models' capabilities and distinguishing them from competitors. However, companies have also found ways to outsmart legacy benchmarking systems, which are often outdated and not designed to measure modern AI capabilities.
- In a separate development, the Indian government launched the Elderly Friendly Office Certification (EFOC) program, which sets technical standards for offices to better accommodate senior citizens. The program includes clear, common standards for inspections and awards offices Gold, Silver, or Bronze certificates.
- The EFOC program addresses common issues faced by senior citizens, such as long queues, stairs without handrails, and forms in small print. This is the first time such standards have been implemented in India.
- Meanwhile, the U.S. government is focusing on advancing AI through the Genesis Mission, a whole-of-government initiative led by the White House Office of Science and Technology Policy. The mission aims to double the productivity and impact of American science and engineering within a decade.
- President Trump, in a social media post, announced new fuel economy standards that roll back Biden-era vehicle rules. The changes are expected to lower car prices but may also increase fuel consumption.
- The Department of Transportation estimates that the new standards will reduce the typical price of a new car by about $930 but could lead to an increase in fuel consumption by about 100 billion gallons through 2050.
What's still developing
- 25% off tickets now Back by popular demand: Save up to $300 on Disrupt Benchmarking has become the industry norm for how AI companies validate their models’ capabilities and, when the metrics swing in their favor, stand out from competitors and advertise their superiority.
- Vals, a startup formed in 2024, says that it is on a mission to fix this very imperfect system.
- “The Sharjah model in the UAE, the World Health Organization’s Age-Friendly Cities network, and the Certified Age-Friendly Employer programme in the United States served as models for EFOC. Technical standards for ramps, signage and toilets follow the national guidelines issued in 2021 under the Rights of Persons with Disabilities Act. This is the first time such standards have been brought together in India,” according to a release.
- “Our contribution to the Genesis Mission exemplifies NIST’s ongoing commitment to advancing U.S. leadership in critical technologies that are vital to America’s economic and national security,” said Under Secretary of Commerce for Standards and Technology and NIST Director Arvind Raman.
- Trump wrote, “These new Standards will take the waste out of building cars in America. That means LOWER PRICES, saving families thousands on a new, beautiful, and safe car,” in a social media post.
- Transportation Secretary Sean Duffy said the administration would officially announce the change on Monday, after Trump said he had just approved new fuel economy standards on Truth Social.
- CNBC said Trump presented the policy change as a boon for both automakers and consumers, but it also said the final standards have not yet been publicly detailed.
- The move ends Biden's electric vehicle mandate and aims to lower car prices.