Nano-jev: Calibrated Cross-Encoder Micro-Decisions for Retrieval-Augmented Generation with Confidence Cascading
Research Article  ·  Published: 16 November 2026
Issue cover
ICCK Transactions on Intelligent Systematics
Volume 3, Issue 4, 2026: 245-
Research Article Free to Read

Nano-jev: Calibrated Cross-Encoder Micro-Decisions for Retrieval-Augmented Generation with Confidence Cascading

1 Department of Electronics and Telecommunications, C. V. Raman Global University, Bhubaneswar 752054, India
2 Department of Computer Science, C. V. Raman Global University, Bhubaneswar 752054, India
* Corresponding Authors: Subham Divakar, [email protected]; Rojalina Priyadarshini, [email protected]
Volume 3, Issue 4
You have access to this article · Limited-Time Free Access

Article Information

Abstract

Retrieval-Augmented Generation (RAG) pipelines repeatedly make small judgments, such as whether a passage is relevant, whether the retrieved context is sufficient and whether an answer is grounded. These are usually delegated to the generating large language model (LLM), at notable latency and cost, or to score thresholds without a probabilistic interpretation. We study nano-jev, a 33.4\,M-parameter cross-encoder that treats relevance (3-way), sufficiency and groundedness (binary) as typed, closed-set classification problems and returns temperature-calibrated probabilities. Each (question, option, context) triple is scored by a single-logit head, which makes predictions invariant to option order. On held-out halves of the HotpotQA, SQuAD~2.0 and MultiNLI validation sets, nano-jev reaches mean accuracies of 0.812 (relevance), 0.843 (sufficiency) and 0.841 (groundedness) over five seeds, comparable to a 1-pass classifier trained on the same data, which is 2.5$\times$ faster. Dedicated checkers such as MiniCheck remain stronger on groundedness (0.866). nano-jev runs 55-93$\times$ faster than a prompted 4\,B-parameter LLM. On unseen datasets (MuSiQue, VitaminC), accuracy drops by 15-20 points and ECE rises from about 0.03 to about 0.17, so target-domain re-calibration is advisable. A confidence cascade that escalates uncertain decisions to the LLM improves in-distribution groundedness (0.868 vs. 0.841, $p=0.02$) with about a quarter of decisions escalated, but trades accuracy for latency out of distribution. In a small end-to-end test, nano-jev relevance filtering improves HotpotQA answer F1 by 2.1 points at about 39ms per question. Code and weights are released as an open-source package.

Graphical Abstract

Nano-jev: Calibrated Cross-Encoder Micro-Decisions for Retrieval-Augmented Generation with Confidence Cascading

Keywords

retrieval-augmented generation cross-encoder decision model calibration confidence cascade natural language inference question answering

Data Availability Statement

The complete nano-jev implementation, including model definitions, training and evaluation scripts, baseline code, calibration routines, random seeds and unit tests, is publicly available at https://github.com/shubham10divakar/nano-jev and can be installed from PyPI with pip install nano-jev. Pre-trained weights (v0.1 and v1.0) are hosted on Hugging Face at https://huggingface.co/sdmlai/nano-jev. All datasets used for training and evaluation are publicly available from their original providers under their respective licences.

Funding

This work was supported without any funding.

Conflicts of Interest

The authors declare no conflicts of interest.

AI Use Statement

The authors declare that ChatGPT-5 was used for language editing in the Introduction and Conclusion sections. The authors have carefully reviewed, revised, and verified the AI-assisted output and take full responsibility for the content of the manuscript.

Ethical Approval and Consent to Participate

Not applicable.

References

  1. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., ... & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33, 9459-9474.
    [Google Scholar]
  2. Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., ... & Kaplan, J. (2022). Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221.
    [CrossRef] [Google Scholar]
  3. Kuhn, L., Gal, Y., & Farquhar, S. (2023). Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. arXiv preprint arXiv:2302.09664.
    [CrossRef] [Google Scholar]
  4. Zha, Y., Yang, Y., Li, R., & Hu, Z. (2023, July). AlignScore: Evaluating factual consistency with a unified alignment function. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers) (pp. 11328-11348).
    [CrossRef] [Google Scholar]
  5. Tang, L., Laban, P., & Durrett, G. (2024). MiniCheck: Efficient fact-checking of LLMs on grounding documents. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (pp. 8818-8847).
    [CrossRef] [Google Scholar]
  6. Saad-Falcon, J., Khattab, O., Potts, C., & Zaharia, M. (2024). ARES: An automated evaluation framework for retrieval-augmented generation systems. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics (pp. 338-354).
    [CrossRef] [Google Scholar]
  7. Es, S., James, J., Espinosa-Anke, L., & Schockaert, S. (2024). RAGAs: Automated evaluation of retrieval augmented generation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations (pp. 150-158).
    [CrossRef] [Google Scholar]
  8. Guu, K., Lee, K., Tung, Z., Pasupat, P., & Chang, M. (2020, November). Retrieval augmented language model pre-training. In International conference on machine learning (pp. 3929-3938). PMLR. https://proceedings.mlr.press/v119/guu20a.html
    [Google Scholar]
  9. Asai, A., Wu, Z., Wang, Y., Sil, A., & Hajishirzi, H. (2024, May). Self-rag: Learning to retrieve, generate, and critique through self-reflection. In International conference on learning representations (Vol. 2024, pp. 9112-9141).
    [Google Scholar]
  10. Yan, S. Q., Gu, J. C., Zhu, Y., & Ling, Z. H. (2024). Corrective retrieval augmented generation. arXiv preprint arXiv:2401.15884.
    [CrossRef] [Google Scholar]
  11. Laban, P., Schnabel, T., Bennett, P. N., & Hearst, M. A. (2022). SummaC: Re-visiting NLI-based models for inconsistency detection in summarization. Transactions of the Association for Computational Linguistics, 10, 163-177.
    [CrossRef] [Google Scholar]
  12. Honovich, O., Aharoni, R., Herzig, J., Taitelbaum, H., Kukliansy, D., Cohen, V., ... & Matias, Y. (2022, July). TRUE: Re-evaluating factual consistency evaluation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 3905-3920).
    [CrossRef] [Google Scholar]
  13. Niu, C., Wu, Y., Zhu, J., Xu, S., Shum, K., Zhong, R., ... & Zhang, T. (2024). RAGTruth: A hallucination corpus for developing trustworthy retrieval-augmented language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (pp. 10862-10878).
    [CrossRef] [Google Scholar]
  14. Nogueira, R., Yang, W., Cho, K., & Lin, J. (2019). Multi-stage document ranking with BERT. arXiv preprint arXiv:1910.14424.
    [CrossRef] [Google Scholar]
  15. Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (pp. 3982-3992).
    [CrossRef] [Google Scholar]
  16. Chen, J., Xiao, S., Zhang, P., Luo, K., Lian, D., & Liu, Z. (2024). M3-Embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In Findings of the Association for Computational Linguistics: ACL 2024 (pp. 2318-2335).
    [CrossRef] [Google Scholar]
  17. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 4171-4186).
    [CrossRef] [Google Scholar]
  18. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., ... & Stoyanov, V. (2019). RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692.
    [CrossRef] [Google Scholar]
  19. He, P., Gao, J., & Chen, W. (2021). Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. arXiv preprint arXiv:2111.09543.
    [CrossRef] [Google Scholar]
  20. Yin, W., Hay, J., & Roth, D. (2019). Benchmarking zero-shot text classification: Datasets, evaluation and entailment approach. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (pp. 3914-3923).
    [CrossRef] [Google Scholar]
  21. Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning (pp. 1321-1330). https://proceedings.mlr.press/v70/guo17a.html
    [Google Scholar]
  22. Platt, J. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers, 10(3), 61-74. https://www.researchgate.net/publication/2594015
    [Google Scholar]
  23. Chen, L., Zaharia, M., & Zou, J. (2023). Frugalgpt: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176.
    [CrossRef] [Google Scholar]
  24. Ding, D., Mallick, A., Wang, C., Sim, R., Mukherjee, S., Rühle, V., ... & Awadallah, A. H. (2024, May). Hybrid llm: Cost-efficient and quality-aware query routing. In International Conference on Learning Representations (Vol. 2024, pp. 41348-41366).
    [Google Scholar]
  25. Leviathan, Y., Kalman, M., & Matias, Y. (2023, July). Fast inference from transformers via speculative decoding. In International conference on machine learning (pp. 19274-19286). PMLR. https://proceedings.mlr.press/v202/leviathan23a.html
    [Google Scholar]
  26. Almeida, D. (2026, September 15). Introducing System One models & Jev. TypeSafe AI Blog. Retrieved from https://typesafe.ai/blog/introducing-system-one-models-and-jev
    [Google Scholar]
  27. Wang, W., Wei, F., Dong, L., Bao, H., Yang, N., & Zhou, M. (2020). MiniLM: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. In Advances in Neural Information Processing Systems (Vol. 33, pp. 5776-5788).
    [Google Scholar]
  28. Liu, D. C., & Nocedal, J. (1989). On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45(1-3), 503-528.
    [CrossRef] [Google Scholar]
  29. Naeini, M. P., Cooper, G., & Hauskrecht, M. (2015). Obtaining well calibrated probabilities using Bayesian binning. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 29, No. 1, pp. 2901-2907).
    [CrossRef] [Google Scholar]
  30. Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018). HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 conference on empirical methods in natural language processing (pp. 2369-2380).
    [CrossRef] [Google Scholar]
  31. Rajpurkar, P., Jia, R., & Liang, P. (2018). Know what you don't know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (pp. 784-789).
    [CrossRef] [Google Scholar]
  32. Williams, A., Nangia, N., & Bowman, S. R. (2018, June). A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long papers) (pp. 1112-1122).
    [CrossRef] [Google Scholar]
  33. Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2022). MuSiQue: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10, 539-554.
    [CrossRef] [Google Scholar]
  34. Schuster, T., Fisch, A., & Barzilay, R. (2021). Get your vitamin C! Robust fact verification with contrastive evidence. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 624-643).
    [CrossRef] [Google Scholar]
  35. Zhang, X., Zhao, J., & LeCun, Y. (2015). Character-level convolutional networks for text classification. In Advances in Neural Information Processing Systems (Vol. 28, pp. 649-657).
    [Google Scholar]
  36. Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A. Y., & Potts, C. (2013). Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (pp. 1631-1642). https://aclanthology.org/D13-1170/
    [Google Scholar]
  37. Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018, June). FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) (pp. 809-819).
    [CrossRef] [Google Scholar]
  38. Nie, Y., Williams, A., Dinan, E., Bansal, M., Weston, J., & Kiela, D. (2020). Adversarial NLI: A new benchmark for natural language understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 4885-4901).
    [CrossRef] [Google Scholar]
  39. Qwen Team. (2025). Qwen3 technical report. arXiv preprint arXiv:2505.09388.
    [CrossRef] [Google Scholar]
  40. Loshchilov, I., & Hutter, F. (2019). Decoupled weight decay regularization. arXiv preprint arXiv:2505.09388.
    [CrossRef] [Google Scholar]
  41. Efron, B., & Tibshirani, R. J. (1994). An Introduction to the Bootstrap. Chapman & Hall/CRC.
    [CrossRef] [Google Scholar]
  42. McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2), 153-157.
    [CrossRef] [Google Scholar]
  43. Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., ... & Snoek, J. (2019). Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems (Vol. 32, pp. 13991-14002).
    [Google Scholar]
  44. Park, S., Bastani, O., Weimer, J., & Lee, I. (2020, June). Calibrated prediction with covariate shift via unsupervised domain adaptation. In International Conference on Artificial Intelligence and Statistics (pp. 3219-3229). PMLR. https://proceedings.mlr.press/v108/park20b.html
    [Google Scholar]

Cite This Article

APA Style
Divakar, S., & Priyadarshini, R. (2026). Nano-jev: Calibrated Cross-Encoder Micro-Decisions for Retrieval-Augmented Generation with Confidence Cascading. ICCK Transactions on Intelligent Systematics, 3(4), 245-260. https://doi.org/10.62762/TIS.2026.197904
Export Citation
RIS Format
Compatible with EndNote, Zotero, Mendeley, and other reference managers
TY  - JOUR
AU  - Divakar, Subham
AU  - Priyadarshini, Rojalina
PY  - 2026
DA  - 2026/11/16
TI  - Nano-jev: Calibrated Cross-Encoder Micro-Decisions for Retrieval-Augmented Generation with Confidence Cascading
JO  - ICCK Transactions on Intelligent Systematics
T2  - ICCK Transactions on Intelligent Systematics
JF  - ICCK Transactions on Intelligent Systematics
VL  - 3
IS  - 4
SP  - 245
EP  - 
DO  - 10.62762/TIS.2026.197904
UR  - https://www.icck.org/article/abs/tis.2026.197904
KW  - retrieval-augmented generation
KW  - cross-encoder
KW  - decision model
KW  - calibration
KW  - confidence cascade
KW  - natural language inference
KW  - question answering
AB  - Retrieval-Augmented Generation (RAG) pipelines repeatedly make small judgments, such as whether a passage is relevant, whether the retrieved context is sufficient and whether an answer is grounded. These are usually delegated to the generating large language model (LLM), at notable latency and cost, or to score thresholds without a probabilistic interpretation. We study nano-jev, a 33.4\,M-parameter cross-encoder that treats relevance (3-way), sufficiency and groundedness (binary) as typed, closed-set classification problems and returns temperature-calibrated probabilities. Each (question, option, context) triple is scored by a single-logit head, which makes predictions invariant to option order. On held-out halves of the HotpotQA, SQuAD~2.0 and MultiNLI validation sets, nano-jev reaches mean accuracies of 0.812 (relevance), 0.843 (sufficiency) and 0.841 (groundedness) over five seeds, comparable to a 1-pass classifier trained on the same data, which is 2.5$\times$ faster. Dedicated checkers such as MiniCheck remain stronger on groundedness (0.866). nano-jev runs 55-93$\times$ faster than a prompted 4\,B-parameter LLM. On unseen datasets (MuSiQue, VitaminC), accuracy drops by 15-20 points and ECE rises from about 0.03 to about 0.17, so target-domain re-calibration is advisable. A confidence cascade that escalates uncertain decisions to the LLM improves in-distribution groundedness (0.868 vs. 0.841, $p=0.02$) with about a quarter of decisions escalated, but trades accuracy for latency out of distribution. In a small end-to-end test, nano-jev relevance filtering improves HotpotQA answer F1 by 2.1 points at about 39ms per question. Code and weights are released as an open-source package.
SN  - 3068-5079
PB  - Institute of Central Computation and Knowledge
LA  - English
ER  - 
BibTeX Format
Compatible with LaTeX, BibTeX, and other reference managers
@article{Divakar2026Nanojev,
  author = {Subham Divakar and Rojalina Priyadarshini},
  title = {Nano-jev: Calibrated Cross-Encoder Micro-Decisions for Retrieval-Augmented Generation with Confidence Cascading},
  journal = {ICCK Transactions on Intelligent Systematics},
  year = {2026},
  volume = {3},
  number = {4},
  pages = {245-},
  doi = {10.62762/TIS.2026.197904},
  url = {https://www.icck.org/article/abs/tis.2026.197904},
  abstract = {Retrieval-Augmented Generation (RAG) pipelines repeatedly make small judgments, such as whether a passage is relevant, whether the retrieved context is sufficient and whether an answer is grounded. These are usually delegated to the generating large language model (LLM), at notable latency and cost, or to score thresholds without a probabilistic interpretation. We study nano-jev, a 33.4\,M-parameter cross-encoder that treats relevance (3-way), sufficiency and groundedness (binary) as typed, closed-set classification problems and returns temperature-calibrated probabilities. Each (question, option, context) triple is scored by a single-logit head, which makes predictions invariant to option order. On held-out halves of the HotpotQA, SQuAD~2.0 and MultiNLI validation sets, nano-jev reaches mean accuracies of 0.812 (relevance), 0.843 (sufficiency) and 0.841 (groundedness) over five seeds, comparable to a 1-pass classifier trained on the same data, which is 2.5\$\times\$ faster. Dedicated checkers such as MiniCheck remain stronger on groundedness (0.866). nano-jev runs 55-93\$\times\$ faster than a prompted 4\,B-parameter LLM. On unseen datasets (MuSiQue, VitaminC), accuracy drops by 15-20 points and ECE rises from about 0.03 to about 0.17, so target-domain re-calibration is advisable. A confidence cascade that escalates uncertain decisions to the LLM improves in-distribution groundedness (0.868 vs. 0.841, \$p=0.02\$) with about a quarter of decisions escalated, but trades accuracy for latency out of distribution. In a small end-to-end test, nano-jev relevance filtering improves HotpotQA answer F1 by 2.1 points at about 39ms per question. Code and weights are released as an open-source package.},
  keywords = {retrieval-augmented generation, cross-encoder, decision model, calibration, confidence cascade, natural language inference, question answering},
  issn = {3068-5079},
  publisher = {Institute of Central Computation and Knowledge}
}

Article Metrics

Citations
Crossref
0
Scopus
0
Views
19
PDF Downloads
0

Publisher's Note

ICCK stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and Permissions

Institute of Central Computation and Knowledge (ICCK) or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
ICCK Transactions on Intelligent Systematics
ICCK Transactions on Intelligent Systematics
ISSN: 3068-5079 (Online) | ISSN: 3069-003X (Print)
Portico
Preserved at
Portico