California's Assembly Joint Hearing Health Committee and Privacy and Consumer Protection Committee convened on May 28, 2025, to address pressing issues surrounding the use of artificial intelligence (AI) in healthcare, particularly its implications for equity and data governance. A key focus of the meeting was the urgent need for oversight and evaluation of AI algorithms, which have been found to exhibit racial biases that can adversely affect patient care.

Experts highlighted a significant market failure, noting that many healthcare systems and practitioners failed to identify these biases in AI tools before their widespread adoption. This oversight underscores the necessity for robust data evaluation practices that can only be achieved through comprehensive oversight mechanisms. The discussions emphasized that biases often emerge only when algorithms are analyzed across large datasets, rather than at the individual patient level.

The meeting also explored the current state of Institutional Review Board (IRB) protocols in relation to AI. While some pathways exist for research studies involving AI, the lack of governance in quality improvement initiatives poses a risk. Many hospitals lack dedicated committees to oversee the deployment of AI technologies, leading to potential ethical and operational challenges.

Panelists expressed particular concern for Medi-Cal populations, emphasizing the disparities in access to advanced AI tools between safety net institutions and wealthier healthcare providers. The enthusiasm for adopting AI among safety net providers was noted, but it was accompanied by apprehension regarding governance and the potential exacerbation of existing inequities.

Recommendations from the panel included a clear focus on addressing data bias and equity, establishing liability frameworks for AI-related issues, and ensuring sustainable funding for both the deployment and governance of AI tools. The discussions underscored the importance of aligning AI development with the specific needs of diverse patient populations, particularly those served by Medi-Cal.

As the meeting concluded, stakeholders were encouraged to prioritize these issues to ensure that AI technologies enhance rather than hinder equitable healthcare access. The next steps involve further discussions with a range of stakeholders, including healthcare associations and community organizations, to develop actionable policies that address these critical challenges.