How to mitigate SQL injection
One of the most efficient ways of securing against SQL injection is Parametrized Statement. Instead of combining the SQL that we want to send to a database with the parameters coming from the user, we treat the SQL and its parameters as 2 separate things, and then we let the database combine them in a secure way.
How to mitigate Prompt injection
One of the most efficient ways of securing against Prompt Injection is Instruction Hierarchy, which means that we train the LLM to allow system prompt to overwrite user prompt, but not the other way around. The problem is that it doesnt always work. Also, we often combine the prompt with the arguments coming from the user just by concatenating them together because the input from the user is not the whole request, but only part of it.
How to mitigate Prompt injection even better
Would separating the request from the arguments change anything?
Here's the problem:
Let's imagine that the instructions about capital letters are coming from the service, and the text added below is coming from the user of that service. I used here a free tier of ChatGPT not to need to come up with longer and less presentable examples. The problem is that the text coming from the user was treated as instructions, not as a parameter that the instructions are about.
Let's try to fix that using the same approach that works for mitigating SQL injections:
That works! The problem I encountered with longer parameters is that by the end of it, the model forgot the instructions. Changing the order of the arguments and the instructions solved that problem though:
Can we do it even better?
I wonder if combining this approach with Instruction Hierarchy would be even more robust. Let's imagine, for example, that apart from the roles: system and user, we would have argument, and we would train the model to always ignore the instructions from the argument, unless the instructions from the role user explicitely say so...