The AI-Specific Penetration Testing Guide provided by CSA outlines essential considerations for conducting penetration tests that involve AI systems, addressing scoping, using API from established providers, self-hosting large language models, fine-tuning training models, setting goals for AI systems tests, fitting AI testing into broader security strategies, and determining testing methodologies from OWASP, MITRE, and NIST guidelines to ensure comprehensive assessment and root cause remediation, emphasizing collaborative efforts, resilience, and maximizing the impact of findings within the agreed scope.
https://cloudsecurityalliance.org/articles/considerations-when-including-ai-implementations-in-penetration-testing