Generative AI Under the Lens: Rigorous Checks for Safety and Reliability
Generative AI has accelerated development across various domains - including software, robotics, autonomous systems, and industrial applications - enabling rapid innovation and productivity gains. However, deploying generative AI-driven systems can introduce safety, security, and reliability vulnerabilities, highlighting the urgent need for rigorous evaluation of their outputs before deployment. Among many evaluation techniques, this workshop will focus on formal and statistical verification approaches.
Generative AI requires careful verification. For example, in software development, 45% of organizations prioritize speed over quality, and 63% deploy code without full testing. While 80% view generative AI as enhancing both speed and quality, studies reveal significant flaws that could compromise safety and security, underscoring the need for efficient yet thorough verification methods across all AI-driven systems.
Generative AI can also support verification in multiple ways, including:
- Automating specification generation from system requirements to accelerate verification workflows.
- Enhancing formal verification tools by guiding proofs, generating counterexamples, and analyzing high-risk areas.
- Optimizing verification productivity and coverage in complex systems, including hardware and software co-design.
- Integrating with statistical methods to improve reliability, uncertainty quantification, and diagnostics.
These AI-assisted techniques reduce human effort, scale verification to complex systems, and enable tasks previously infeasible, while maintaining rigorous guarantees.
The workshop will convene researchers, practitioners, and industry experts to explore the following research question: How can generative AI-assisted verification techniques, including formal and statistical approaches, ensure that AI-driven systems are rigorously checked for safety, reliability, and security across diverse domains?
Participants will explore cutting-edge methods and collaborative opportunities to advance trustworthy and reliable generative AI deployment.
