LLMs are popular, and like any popular tech deployed rapidly, there are many security holes and risks with LLMs as they are deployed today. Security risks present in areas like reputational harm, privacy leaks, and misinformation. The big LLM providers have generally been very slow or even ineffectual at responding to these threats. In parallel, both the NLP and Security communities have started working together to map out the new, expansive area of LLM Security, through initiatives like the OWASP Top 10 for LLM, or the US NIST Generative AI working group. Meanwhile, a huge and diverse grass-roots movement of LLM probing has developed, often outpacing measures taken by providers with a creative and sometimes entertaining set of exploits.

In this talk, I'll give a tour of contemporary LLM security, detailing exploits and mitigations. We'll touch upon LLM design, red teaming, risks, attack tactics, data exfiltration, multiple flavours of prompt injection, sponge attacks, and some of the more unusual vulnerabilities found in the wild. I'll introduce tools for auditing and vulnerability scanning LLMs, as well as potential mitigations, so that we can learn how to use this new technology safely and security. If you want to read up in advance, check llmsecurity.net.