Many organisations are adding large language models to their products and internal tools, often with access to customer data, documents and other systems. The model itself is only one part of the risk. The surrounding application decides what the model can see, what it can do, and what happens to its output, and that is where most of the serious issues are found. We offer testing of LLM-based applications of all kinds, from a public chatbot on a website to an internal agent that can read email and raise tickets. This service is ideal for organisations that are deploying or already running an LLM feature and want to understand its exposure before an attacker does.
LLM Application Methodology Summary
Application Mapping
We will review the full attack surface before continuing onto the security assessment. For an LLM application this means establishing which models are in use, what the system prompt and guardrails are intended to do, which data sources the model can retrieve from, which tools, plugins or APIs it can call, and how the application uses what the model returns. This can include reviewing the assessment scope to ensure that no components have been unintentionally missed.
Application Vulnerability Discovery
We review the application for the presence of a range of vulnerabilities, including but not limited to those covered by awareness documents such as the OWASP Top 10 for LLM Applications. The following list gives an indication of the types of vulnerability that can be discovered through this type of engagement:
- Direct and indirect prompt injection
- Sensitive information disclosure
- Supply chain weaknesses in models, plugins and datasets
- Data and model poisoning
- Improper output handling
- Excessive agency
- System prompt leakage
- Vector and embedding weaknesses
- Misinformation
- Unbounded consumption
- Guardrail and content filter bypass
- Broken access control between users, tenants and data sources
- Business logic issues
Analysis and Exploitation
Where a vulnerability is discovered our testing report will include a full breakdown of the potential for exploitation thereby removing false positives, removing the guesswork from grading vulnerability risks, and giving the steps to replicate the vulnerability to ensure that your technical teams fully understand each issue. For prompt-based issues this includes the exact inputs used, so that fixes can be verified against them.
Supporting Application and API Assessment
An LLM feature almost always sits inside a conventional web application or API. Where requested we can include that layer within the assessment, covering the same areas as a Web Application Penetration Test: authentication, access control, injection, session handling and the configuration of the hosting infrastructure.
Remediation
We don’t just focus on vulnerability discovery, but we also give significant detail on remediating discovered vulnerabilities and importantly, hardening systems against exploitation. For LLM applications this covers the design of prompts and guardrails, the permissions given to the model and its tools, and the handling of model output, as well as the conventional fixes to the application around it.