A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots
Research Article  ·  Published: 25 September 2026
Issue cover
ICCK Transactions on Information Security and Cryptography
Volume 2, Issue 3, 2026: 192-215
Research Article Free to Read

A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots

1 Department of Computer Science, COMSATS University Islamabad, Lahore 54000, Pakistan
2 Institute of Biomedicine, University of Turku, Turku FI-20014, Finland
3 Sparkverse AI Ltd, Bradford BD1 1AA, West Yorkshire, United Kingdom
4 School of Science and Technology, Georgia Gwinnett College, Lawrenceville 30043, Georgia, United States
* Corresponding Author: Nisar Ahmed, [email protected]
Volume 2, Issue 3
You have access to this article · Limited-Time Free Access

Article Information

Abstract

Prompt injection is ranked as the most critical vulnerability in large language model (LLM) deployments by the OWASP Top 10 for LLM Applications, yet existing defenses operate at isolated pipeline stages and remain incomplete. Input filters cannot inspect retrieved documents, while output monitors cannot prevent malicious payloads from reaching the model. Consequently, retrieval-augmented generation (RAG) chatbots remain vulnerable to indirect injection, where a poisoned knowledge-base document compromises every user whose query retrieves it. We present a three-layer framework that intercepts both direct and indirect prompt injection throughout the inference pipeline. Layer 1 screens user input using a rule-based pattern library and a fine-tuned semantic anomaly classifier. Layer 2 enforces a provenance-based instruction hierarchy during context assembly, preventing retrieved content from overriding operator policy. Layer 3 audits model output using a policy rule engine and semantic drift detector before delivery. A continuous audit loop aggregates structured logs and supports retraining to adapt the classifier to emerging attack patterns. The framework is target-model agnostic: it treats the protected LLM as a black box and deploys as middleware without modifying it, while relying on lightweight auxiliary models (a MiniLM sentence encoder and Llama Guard~2) for detection. Evaluation on 5,080 samples across GPT-4o, Llama 3, and Mistral 7B - restricted to single-turn and non-adaptive attacks - shows that the framework reduces Attack Success Rate (ASR) from 71.4% to 11.3%, outperforming the best single-layer baseline by 27.3 percentage points and the evaluated NeMo Guardrails configuration by 23.8 percentage points, while maintaining a 4.8% false positive rate and a median latency overhead of 61.2 ms. Ablation studies show that all three layers contribute and that the full framework achieves a lower attack success rate than would be expected if the layers acted independently.

Graphical Abstract

A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots

Keywords

prompt injection detection retrieval-augmented generation large language models chatbot security large language model guardrails

Data Availability Statement

The public benchmark datasets used in this study-PromptBench, BIPIA, and MS~MARCO-are available from their respective original sources under their respective licences. The 788 manually curated adversarial samples and the 1,440 GPT-4o-generated paraphrase augmentations were constructed by the authors and are available from the corresponding author upon reasonable request. The implementation code and related resources are publicly available at https://github.com/nisarahmedrana/Prompt_Injection_Attack/.

Funding

This work was supported without any funding.

Conflicts of Interest

Nisar Ahmed served as an Associate Editor of the ICCK Transactions on Information Security and Cryptography at the time of manuscript submission. To ensure the integrity of the peer-review process, Nisar Ahmed was excluded from the editorial handling, peer review, and final decision-making for this manuscript; these responsibilities were handled independently by another editor. Ali Hassan is affiliated with Sparkverse AI Ltd., Bradford BD1 1AA, West Yorkshire, United Kingdom. The authors confirm that this commercial affiliation did not influence the study design, data collection, analysis, interpretation of results, or the decision to submit for publication. The remaining authors declare no competing interests.

AI Use Statement

The authors declare that GPT-4o, developed by OpenAI, was used to generate paraphrased variants of adversarial samples for dataset augmentation and to assist with automated scoring of Instruction Override goal achievement. GPT-4o was not used to draft the text of this manuscript. The authors carefully reviewed and verified the AI-generated outputs and take full responsibility for the quality, accuracy, and integrity of the AI-assisted research materials and the manuscript.

Ethical Approval and Consent to Participate

Not applicable.

References

  1. OpenAI. (2024). GPT-4o System Card. Retrieved from https://openai.com/index/gpt-4o-system-card
    [Google Scholar]
  2. Meta AI. (2024). Llama 3 Model Card. Retrieved from https://ai.meta.com/blog/meta-llama-3/
    [Google Scholar]
  3. Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., de las Casas, D., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., Lavaud, L. R., Lachaux, M.-A., Stock, P., Le Scao, T., Lavril, T., Wang, T., Lacroix, T., & El Sayed, W. (2023). Mistral 7B. arXiv preprint arXiv:2310.06825.
    [CrossRef] [Google Scholar]
  4. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., ... & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33, 9459-9474.
    [Google Scholar]
  5. OWASP Foundation. (2023). OWASP Top 10 for Large Language Model Applications. Retrieved from https://owasp.org/www-project-top-10-for-large-language-model-applications/
    [Google Scholar]
  6. Perez, F., & Ribeiro, I. (2022). Ignore previous prompt: Attack techniques for language models. arXiv preprint arXiv:2211.09527.
    [CrossRef] [Google Scholar]
  7. Willison, S. (2022). Prompt Injection Attacks Against GPT-3. Retrieved from https://simonwillison.net/2022/Sep/12/prompt-injection/
    [Google Scholar]
  8. Halfond, W. G., Viegas, J., & Orso, A. (2006, March). A Classification of SQL Injection Attacks and Countermeasures. In Proceedings of the IEEE International Symposium on Secure Software Engineering (pp. 13–15). IEEE. https://cgi.cse.unsw.edu.au/~meyden/3441/halfond.pdf
    [Google Scholar]
  9. Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023, November). Not what you've signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM workshop on artificial intelligence and security (pp. 79-90).
    [CrossRef] [Google Scholar]
  10. Branch, H. J., Cefalu, J. R., McHugh, J., Hujer, L., Bahl, A., Iglesias, D. D. C., ... & Darwishi, R. (2022). Evaluating the susceptibility of pre-trained language models via handcrafted adversarial examples. arXiv preprint arXiv:2209.02128.
    [CrossRef] [Google Scholar]
  11. Liu, Y., Deng, G., Li, Y., Wang, K., Wang, Z., Wang, X., ... & Liu, Y. (2023). Prompt injection attack against llm-integrated applications. arXiv preprint arXiv:2306.05499.
    [CrossRef] [Google Scholar]
  12. Wallace, E., Xiao, K., Leike, R., Weng, L., Heidecke, J., & Beutel, A. (2024). The instruction hierarchy: Training llms to prioritize privileged instructions. arXiv preprint arXiv:2404.13208.
    [CrossRef] [Google Scholar]
  13. Rebedea, T., Dinu, R., Sreedhar, M. N., Parisien, C., & Cohen, J. (2023, December). Nemo guardrails: A toolkit for controllable and safe llm applications with programmable rails. In Proceedings of the 2023 conference on empirical methods in natural language processing: system demonstrations (pp. 431-445).
    [CrossRef] [Google Scholar]
  14. Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y., ... & Khabsa, M. (2023). Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674.
    [CrossRef] [Google Scholar]
  15. Phute, M., Helbling, A., Hull, M., Peng, S., Szyller, S., Cornelius, C., & Chau, D. H. (2023). Llm self defense: By self examination, llms know they are being tricked. arXiv preprint arXiv:2308.07308.
    [CrossRef] [Google Scholar]
  16. Kumar, A., Agarwal, C., Srinivas, S., Li, A. J., Feizi, S., & Lakkaraju, H. (2023). Certifying llm safety against adversarial prompting. arXiv preprint arXiv:2309.02705.
    [CrossRef] [Google Scholar]
  17. Yi, J., Xie, Y., Zhu, B., Kiciman, E., Sun, G., Xie, X., & Wu, F. (2025, July). Benchmarking and defending against indirect prompt injection attacks on large language models. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1 (pp. 1809-1820).
    [CrossRef] [Google Scholar]
  18. Liu, Y., Deng, G., Xu, Z., Li, Y., Zheng, Y., Zhang, Y., ... & Liu, Y. (2023). Jailbreaking chatgpt via prompt engineering: An empirical study. arXiv preprint arXiv:2305.13860.
    [CrossRef] [Google Scholar]
  19. Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z., & Fredrikson, M. (2023). Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043.
    [CrossRef] [Google Scholar]
  20. Jain, N., Schwarzschild, A., Wen, Y., Somepalli, G., Kirchenbauer, J., Chiang, P. Y., ... & Goldstein, T. (2023). Baseline defenses for adversarial attacks against aligned language models. arXiv preprint arXiv:2309.00614.
    [CrossRef] [Google Scholar]
  21. Robey, A., Wong, E., Hassani, H., & Pappas, G. J. (2023). Smoothllm: Defending large language models against jailbreaking attacks. arXiv preprint arXiv:2310.03684.
    [CrossRef] [Google Scholar]
  22. Xie, Y., Yi, J., Shao, J., Curl, J., Lyu, L., Chen, Q., Xie, X., & Wu, F. (2023). Defending ChatGPT Against Jailbreak Attack via Self-Reminder. Nature Machine Intelligence, 5(12), 1486--1496.
    [CrossRef] [Google Scholar]
  23. Chen, S., Piet, J., Sitawarin, C., & Wagner, D. (2025). \{StruQ\: Defending against prompt injection with structured queries. In 34th USENIX Security Symposium (USENIX Security 25) (pp. 2383-2400). https://www.usenix.org/conference/usenixsecurity25/presentation/chen-sizhe
    [Google Scholar]
  24. Hines, K., Lopez, G., Hall, M., Zarfati, F., Zunger, Y., & Kiciman, E. (2024). Defending Against Indirect Prompt Injection Attacks With Spotlighting. arXiv preprint arXiv:2403.14720.
    [CrossRef] [Google Scholar]
  25. Zhu, K., Wang, J., Zhou, J., Wang, Z., Chen, H., Wang, Y., ... & Xie, X. (2023, November). Promptrobust: Towards evaluating the robustness of large language models on adversarial prompts. In Proceedings of the 1st ACM workshop on large AI systems and models with privacy and safety analysis (pp. 57-68).
    [CrossRef] [Google Scholar]
  26. Schulhoff, S., Pinto, J., Khan, A., Bouchard, L. F., Si, C., Anati, S., ... & Boyd-Graber, J. L. (2023, December). Ignore this title and HackAPrompt: Exposing systemic vulnerabilities of LLMs through a global prompt hacking competition. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 4945-4977).
    [CrossRef] [Google Scholar]
  27. Shostack, A. (2014). Threat Modeling: Designing for Security. Wiley. https://dl.acm.org/doi/book/10.5555/2829295
    [Google Scholar]
  28. Zhang, Y., & Ippolito, D. Prompts Should not be Seen as Secrets: Systematically Measuring Prompt Extraction Attack Success (2023). arXiv preprint arXiv:2307.06865.
    [Google Scholar]
  29. Zou, W., Geng, R., Wang, B., & Jia, J. (2025). PoisonedRAG: Knowledge corruption attacks to {Retrieval-Augmented generation of large language models. In 34th USENIX Security Symposium (USENIX Security 25) (pp. 3827-3844). https://www.usenix.org/conference/usenixsecurity25/presentation/zou-poisonedrag
    [Google Scholar]
  30. Bagdasaryan, E., Hsieh, T. Y., Nassi, B., & Shmatikov, V. (2023). Abusing images and sounds for indirect instruction injection in multi-modal LLMs. arXiv preprint arXiv:2307.10490.
    [CrossRef] [Google Scholar]
  31. Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., ... & Kaplan, J. (2022). Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073.
    [CrossRef] [Google Scholar]
  32. Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., ... & Yih, W. T. (2020, November). Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP) (pp. 6769-6781).
    [CrossRef] [Google Scholar]
  33. Johnson, J., Douze, M., & Jégou, H. (2019). Billion-scale similarity search with GPUs. IEEE transactions on big data, 7(3), 535-547.
    [CrossRef] [Google Scholar]
  34. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., ... & Rush, A. M. (2019). Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771.
    [CrossRef] [Google Scholar]
  35. Reimers, N., & Gurevych, I. (2019, November). Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) (pp. 3982-3992).
    [CrossRef] [Google Scholar]
  36. Loshchilov, I., & Hutter, F. (2017). Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101.
    [CrossRef] [Google Scholar]
  37. Meta AI. (2024). Llama Guard 2: Safeguarding Human-AI Conversations. Retrieved from https://github.com/meta-llama/PurpleLlama/blob/main/Llama-Guard2/MODEL_CARD.md
    [Google Scholar]
  38. Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R., & Deng, L. (2016). MS MARCO: A human generated machine reading comprehension dataset. OpenReview. https://openreview.net/forum?id=Hk1iOLcle
    [Google Scholar]
  39. Zheng, L., Chiang, W. L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., ... & Stoica, I. (2023). Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36, 46595-46623.
    [CrossRef] [Google Scholar]
  40. Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019). Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675.
    [CrossRef] [Google Scholar]
  41. Carlini, N., & Wagner, D. (2017, May). Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp) (pp. 39-57). IEEE.
    [CrossRef] [Google Scholar]

Cite This Article

APA Style
Saleem, G., Ahmed, N., Zaman, M. I., Hassan, A., & Mujahid, U. (2026). A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots. ICCK Transactions on Information Security and Cryptography, 2(3), 192-215. https://doi.org/10.62762/TISC.2026.712476
Export Citation
RIS Format
Compatible with EndNote, Zotero, Mendeley, and other reference managers
TY  - JOUR
AU  - Saleem, Gulshan
AU  - Ahmed, Nisar
AU  - Zaman, Muhammad Imran
AU  - Hassan, Ali
AU  - Mujahid, Umar
PY  - 2026
DA  - 2026/09/25
TI  - A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots
JO  - ICCK Transactions on Information Security and Cryptography
T2  - ICCK Transactions on Information Security and Cryptography
JF  - ICCK Transactions on Information Security and Cryptography
VL  - 2
IS  - 3
SP  - 192
EP  - 215
DO  - 10.62762/TISC.2026.712476
UR  - https://www.icck.org/article/abs/TISC.2026.712476
KW  - prompt injection detection
KW  - retrieval-augmented
KW  - generation
KW  - large language models
KW  - chatbot security
KW  - large language model guardrails
AB  - Prompt injection is ranked as the most critical vulnerability in large language model (LLM) deployments by the OWASP Top 10 for LLM Applications, yet existing defenses operate at isolated pipeline stages and remain incomplete. Input filters cannot inspect retrieved documents, while output monitors cannot prevent malicious payloads from reaching the model. Consequently, retrieval-augmented generation (RAG) chatbots remain vulnerable to indirect injection, where a poisoned knowledge-base document compromises every user whose query retrieves it. We present a three-layer framework that intercepts both direct and indirect prompt injection throughout the inference pipeline. Layer 1 screens user input using a rule-based pattern library and a fine-tuned semantic anomaly classifier. Layer 2 enforces a provenance-based instruction hierarchy during context assembly, preventing retrieved content from overriding operator policy. Layer 3 audits model output using a policy rule engine and semantic drift detector before delivery. A continuous audit loop aggregates structured logs and supports retraining to adapt the classifier to emerging attack patterns. The framework is target-model agnostic: it treats the protected LLM as a black box and deploys as middleware without modifying it, while relying on lightweight auxiliary models (a MiniLM sentence encoder and Llama Guard~2) for detection. Evaluation on 5,080 samples across GPT-4o, Llama 3, and Mistral 7B - restricted to single-turn and non-adaptive attacks - shows that the framework reduces Attack Success Rate (ASR) from 71.4% to 11.3%, outperforming the best single-layer baseline by 27.3 percentage points and the evaluated NeMo Guardrails configuration by 23.8 percentage points, while maintaining a 4.8% false positive rate and a median latency overhead of 61.2 ms. Ablation studies show that all three layers contribute and that the full framework achieves a lower attack success rate than would be expected if the layers acted independently.
SN  - 3070-2429
PB  - Institute of Central Computation and Knowledge
LA  - English
ER  - 
BibTeX Format
Compatible with LaTeX, BibTeX, and other reference managers
@article{Saleem2026A,
  author = {Gulshan Saleem and Nisar Ahmed and Muhammad Imran Zaman and Ali Hassan and Umar Mujahid},
  title = {A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots},
  journal = {ICCK Transactions on Information Security and Cryptography},
  year = {2026},
  volume = {2},
  number = {3},
  pages = {192-215},
  doi = {10.62762/TISC.2026.712476},
  url = {https://www.icck.org/article/abs/TISC.2026.712476},
  abstract = {Prompt injection is ranked as the most critical vulnerability in large language model (LLM) deployments by the OWASP Top 10 for LLM Applications, yet existing defenses operate at isolated pipeline stages and remain incomplete. Input filters cannot inspect retrieved documents, while output monitors cannot prevent malicious payloads from reaching the model. Consequently, retrieval-augmented generation (RAG) chatbots remain vulnerable to indirect injection, where a poisoned knowledge-base document compromises every user whose query retrieves it. We present a three-layer framework that intercepts both direct and indirect prompt injection throughout the inference pipeline. Layer 1 screens user input using a rule-based pattern library and a fine-tuned semantic anomaly classifier. Layer 2 enforces a provenance-based instruction hierarchy during context assembly, preventing retrieved content from overriding operator policy. Layer 3 audits model output using a policy rule engine and semantic drift detector before delivery. A continuous audit loop aggregates structured logs and supports retraining to adapt the classifier to emerging attack patterns. The framework is target-model agnostic: it treats the protected LLM as a black box and deploys as middleware without modifying it, while relying on lightweight auxiliary models (a MiniLM sentence encoder and Llama Guard~2) for detection. Evaluation on 5,080 samples across GPT-4o, Llama 3, and Mistral 7B - restricted to single-turn and non-adaptive attacks - shows that the framework reduces Attack Success Rate (ASR) from 71.4\% to 11.3\%, outperforming the best single-layer baseline by 27.3 percentage points and the evaluated NeMo Guardrails configuration by 23.8 percentage points, while maintaining a 4.8\% false positive rate and a median latency overhead of 61.2 ms. Ablation studies show that all three layers contribute and that the full framework achieves a lower attack success rate than would be expected if the layers acted independently.},
  keywords = {prompt injection detection, retrieval-augmented, generation, large language models, chatbot security, large language model guardrails},
  issn = {3070-2429},
  publisher = {Institute of Central Computation and Knowledge}
}

Article Metrics

Citations
Crossref
0
Scopus
0
Views
24
PDF Downloads
5

Publisher's Note

ICCK stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and Permissions

Institute of Central Computation and Knowledge (ICCK) or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
ICCK Transactions on Information Security and Cryptography
ICCK Transactions on Information Security and Cryptography
ISSN: 3070-2429 (Online)
Portico
Preserved at
Portico