Journal of Reliable and Secure Computing Home About Editors Current Issue
Federated Fine-Tuning of Large Language Models on Resource-Constrained Clients: Technical Approaches, Resource Costs, and Applicability
Review Article  ·  Published: 20 September 2026
Issue cover
Journal of Reliable and Secure Computing
Volume 2, Issue 3, 2026: 194-211
Review Article Open Access

Federated Fine-Tuning of Large Language Models on Resource-Constrained Clients: Technical Approaches, Resource Costs, and Applicability

by
Yunheng Shen Yunheng Shen 1,2
Xiao Liu Xiao Liu 3
Yang Yang Yang Yang 4 *
Hairong Lv Hairong Lv 1 * ORCID
1 Department of Automation, Tsinghua University, Beijing 100084, China
2 JD.com, Inc., Beijing 101111, China
3 School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu 610054, China
4 School of Computing and Information Systems, Singapore Management University, Singapore 188065, Singapore
* Corresponding Authors: Yang Yang, [email protected]; Hairong Lv, [email protected]
Volume 2, Issue 3
Volume 2, Issue 3 2026
Received: 27 July 2026 Revised: 08 September 2026 Accepted: 15 September 2026 Published: 20 September 2026 CrossMark
Download PDF 1.13 MB Cite Metrics
DOI: 10.62762/JRSC.2026.225854
ARK: ark:/57805/jrsc.2026.225854
You have full access to this open access article · CC BY 4.0 License

Article Information

Published in Journal of Reliable and Secure Computing
Volume/Issue Volume 2, Issue 3, 2026
Pages 194-211

Abstract

Federated fine-tuning adapts language models to distributed data and is widely adopted as a privacy-preserving alternative to centralized training, yet constrained clients must still store model weights and training states, execute updates, and communicate with a server. This review examines four composable routes---parameter-efficient and quantized adaptation, backpropagation-free adaptation, proxy or submodel adaptation, and split federated adaptation---through a common framework that traces the objects each endpoint retains, exchanges, and discloses. When a client retains the complete base, reducing adapter state leaves a base-storage floor; boundary communication depends on input dimensions, sequence length, and interaction frequency, so parameter count alone is insufficient. Keeping data local constrains the raw dataset but not the transmitted object: adapter updates, boundary activations, logits, and cached embeddings expose different surfaces, and the encryption, secure-aggregation, or noise mechanisms that protect them add to the same budgets. Server weight sharing removes duplication but retains private states and caches, and combining routes requires recalculating these costs for the resulting protocol. The analysis identifies where savings arise, where costs move, what each route discloses, and which training, deployment, and trustworthiness conditions govern its applicability.

Graphical Abstract

Federated Fine-Tuning of Large Language Models on Resource-Constrained Clients: Technical Approaches, Resource Costs, and Applicability

Keywords

federated learning large language models privacy-preserving computation trustworthy AI parameter-efficient fine-tuning zeroth-order optimization proxy models split learning secure aggregation resource accounting

Data Availability Statement

Not applicable.

Funding

This work was supported in part by the National Natural Science Foundation of China under Grant 62372110, and in part by the Fujian Provincial Natural Science Foundation under Grant 2023J02008.

Conflicts of Interest

Yunheng Shen is affiliated with JD.com, Inc., Beijing 101111, China. The authors declare that this affiliation had no influence on the study design, data collection, analysis, interpretation of the results, or the decision to publish. Yang Yang served as an Associate Editor of the Journal of Reliable and Secure Computing at the time of manuscript submission. To ensure the integrity and impartiality of the peer-review process, Yang Yang was not involved in the editorial handling, peer review, or decision-making for this manuscript, which was handled independently by another editor. The authors declare no other conflicts of interest.

AI Use Statement

The authors declare that ChatGPT-5.6 Sol was used for drafting and language editing of the manuscript. The authors have carefully reviewed, revised, and verified the AI-assisted output and take full responsibility for the content of the manuscript.

Ethical Approval and Consent to Participate

Not applicable.

References

  1. Woisetschläger, H., Isenko, A., Wang, S., Mayer, R., & Jacobsen, H. A. (2024). A survey on efficient federated learning methods for foundation model training. arXiv preprint arXiv:2401.04472.
    [CrossRef] [Google Scholar]
  2. Yan, N., Su, Y., Deng, Y., & Schober, R. (2025). Federated fine-tuning of LLMs: Framework comparison and research directions. IEEE Communications Magazine, 63(10), 52--58.
    [CrossRef] [Google Scholar]
  3. Wu, Y., Tian, C., Li, J., Sun, H., Tam, K., Zhou, Z., ... & Xu, C. (2025). A survey on federated fine-tuning of large language models. arXiv preprint arXiv:2503.12016.
    [CrossRef] [Google Scholar]
  4. Guo, T., Wang, J., Huo, F., Cui, L., Guo, S., Gui, J., & Tao, D. (2025). On the Evolution of Federated Post-Training Large Language Models: A Model Accessibility View. arXiv preprint arXiv:2508.16261.
    [CrossRef] [Google Scholar]
  5. Yang, Y., Long, G., Lu, Q., Zhu, L., Jiang, J., & Zhang, C. (2025). Federated low-rank adaptation for foundation models: A survey. Proceedings of the 34th International Joint Conference on Artificial Intelligence (IJCAI), 10779--10787.
    [CrossRef] [Google Scholar]
  6. Bian, J., Peng, Y., Wang, L., Huang, Y., & Xu, J. (2025). A survey on parameter-efficient fine-tuning for foundation models in federated learning. arXiv preprint arXiv:2504.21099.
    [CrossRef] [Google Scholar]
  7. Yao, Y., Zhang, J., Wu, J., Huang, C., Xia, Y., Yu, T., ... & Joe-Wong, C. (2026). Federated large language models: Current progress and future directions. In Pacific-Asia Conference on Knowledge Discovery and Data Mining (pp. 203-221). Springer, Singapore.
    [CrossRef] [Google Scholar]
  8. Liu, Z., Wang, Y., Wang, R., Tang, X., & Wu, S. (2026). A Survey on Split Learning for LLM Fine-Tuning: Models, Systems, and Privacy Optimizations. arXiv preprint arXiv:2604.24468.
    [CrossRef] [Google Scholar]
  9. Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., ... & Chen, W. (2021). Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685.
    [CrossRef] [Google Scholar]
  10. Dettmers, T., Pagnoni, A., Holtzman, A., & Zettlemoyer, L. (2023). Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems, 36, 10088-10115.
    [CrossRef] [Google Scholar]
  11. Cho, Y. J., Liu, L., Xu, Z., Fahrezi, A., & Joshi, G. (2024). Heterogeneous LoRA for federated fine-tuning of on-device foundation models. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 12903--12913.
    [CrossRef] [Google Scholar]
  12. Wang, Z., Shen, Z., He, Y., Sun, G., Wang, H., Lyu, L., & Li, A. (2024). Flora: Federated fine-tuning large language models with heterogeneous low-rank adaptations. Advances in Neural Information Processing Systems, 37, 22513-22533.
    [CrossRef] [Google Scholar]
  13. Bai, J., Chen, D., Qian, B., Yao, L., & Li, Y. (2024). Federated fine-tuning of large language models under heterogeneous tasks and client resources. Advances in Neural Information Processing Systems, 37, 14457-14483.
    [CrossRef] [Google Scholar]
  14. Li, S., & Voigt, T. (2026). FraQ: Efficient Coordinate-Space Recompression for Federated Low-Rank Adaptation. arXiv preprint arXiv:2608.03605.
    [CrossRef] [Google Scholar]
  15. Sun, Y., Li, Z., Li, Y., & Ding, B. (2024, May). Improving lora in privacy-preserving federated learning. In International conference on learning representations (Vol. 2024, pp. 17978-17994).
    [Google Scholar]
  16. Babakniya, S., Elkordy, A. R., Ezzeldin, Y. H., Liu, Q., Song, K. B., El-Khamy, M., & Avestimehr, S. (2023). Slora: Federated parameter efficient fine-tuning of language models. arXiv preprint arXiv:2308.06522.
    [CrossRef] [Google Scholar]
  17. Zhang, Z., Hu, R., Liu, P., & Xu, J. (2024). Fed-pilot: Optimizing lora allocation for efficient federated fine-tuning with heterogeneous clients. arXiv preprint arXiv:2410.10200.
    [CrossRef] [Google Scholar]
  18. Su, Y., Yan, N., Deng, Y., Dohler, M., & Schober, R. (2026). HAFLQ: Heterogeneous Adaptive Federated LoRA Fine-tuned LLM with Quantization. IEEE Transactions on Wireless Communications, 25, 20054-20070.
    [CrossRef] [Google Scholar]
  19. JianHao, Z., Lv, C., Wang, X., Wu, M., Liu, W., Li, T., ... & Huang, X. J. (2024, November). Promoting data and model privacy in federated learning through quantized lora. In Findings of the Association for Computational Linguistics: EMNLP 2024 (pp. 10501-10512).
    [CrossRef] [Google Scholar]
  20. Malladi, S., Gao, T., Nichani, E., Damian, A., Lee, J. D., Chen, D., & Arora, S. (2023). Fine-tuning language models with just forward passes. Advances in Neural Information Processing Systems, 36, 53038-53075.
    [CrossRef] [Google Scholar]
  21. Xu, M., Cai, D., Wu, Y., Li, X., & Wang, S. (2024). FwdLLM: Efficient federated finetuning of large language models with perturbed inferences. In 2024 USENIX Annual Technical Conference (USENIX ATC 24) (pp. 579-596). https://www.usenix.org/conference/atc24/presentation/xu-mengwei
    [Google Scholar]
  22. Qin, Z., Chen, D., Qian, B., Ding, B., Li, Y., & Deng, S. (2023). Federated full-parameter tuning of billion-sized language models with communication cost under 18 kilobytes. arXiv preprint arXiv:2312.06353.
    [CrossRef] [Google Scholar]
  23. Li, Z., Ying, B., Liu, Z., Dong, C., & Yang, H. (2025). Achieving dimension-free communication in federated learning via zeroth-order optimization. Proceedings of the International Conference on Learning Representations (ICLR), 23769-23803.
    [Google Scholar]
  24. Ling, Z., Chen, D., Yao, L., Li, Y., & Shen, Y. (2024, August). On the convergence of zeroth-order federated tuning for large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 1827-1838).
    [CrossRef] [Google Scholar]
  25. Ahmed, M. A., Pfeiffer, K., Khalili, R., Khdr, H., & Henkel, J. (2025). Efficient Zeroth-Order Federated Finetuning of Language Models on Resource-Constrained Devices. arXiv preprint arXiv:2502.10239.
    [CrossRef] [Google Scholar]
  26. Sun, J., Xu, Z., Yin, H., Yang, D., Xu, D., Chen, Y., & Roth, H. R. (2023). Fedbpt: Efficient federated black-box prompt tuning for large language models. arXiv preprint arXiv:2310.01467.
    [CrossRef] [Google Scholar]
  27. Yang, G., Wu, T., Guo, Y., Sun, Y., & Gong, Y. (2026, May). FedKRSO: Communication and memory efficient federated fine-tuning of large language models. In IEEE INFOCOM 2026-IEEE Conference on Computer Communications (pp. 1-10). IEEE.
    [CrossRef] [Google Scholar]
  28. Shu, Y., Hu, W., Ng, S. K., Low, B. K. H., & Yu, F. R. (2024). Ferret: Federated full-parameter tuning at scale for large language models. arXiv preprint arXiv:2409.06277.
    [CrossRef] [Google Scholar]
  29. Xiao, G., Lin, J., & Han, S. (2023). Offsite-tuning: Transfer learning without full model. arXiv preprint arXiv:2302.04870.
    [CrossRef] [Google Scholar]
  30. Wu, F., Li, Z., Li, Y., Ding, B., & Gao, J. (2024, August). Fedbiot: Llm local fine-tuning in federated learning without full model. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining (pp. 3345-3355).
    [CrossRef] [Google Scholar]
  31. Peng, Z., Fan, X., Chen, Y., Wang, Z., Pan, S., Wen, C., Zhang, R., & Wang, C. (2024). FedPFT: Federated proxy fine-tuning of foundation models. Proceedings of the 33rd International Joint Conference on Artificial Intelligence (IJCAI), 4806--4814.
    [CrossRef] [Google Scholar]
  32. Wu, Y., Li, J., Tian, C., Guo, Z., & Li, L. (2025). Memory-efficient federated fine-tuning of large language models via layer pruning. arXiv preprint arXiv:2508.17209.
    [CrossRef] [Google Scholar]
  33. Zhu, M., Guo, S., Zhou, P., Ning, Y., Han, C., & Qiao, D. (2025). FedSODA: Federated fine-tuning of LLMs via similarity group pruning and orchestrated distillation alignment. ECAI 2025, Frontiers in Artificial Intelligence and Applications, 413. IOS Press.
    [CrossRef] [Google Scholar]
  34. Fan, T., Ma, G., Song, Y., Fan, L., Chen, K., & Yang, Q. (2026, July). FedProxy: Federated Fine-Tuning of LLMs via Proxy SLMs and Heterogeneity-Aware Fusion. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 17482-17497).
    [CrossRef] [Google Scholar]
  35. Gao, Z., Zhang, Y., Zhang, Z., Gong, Y., & Guo, Y. (2024). FedPT: federated proxy-tuning of large language models on resource-constrained edge devices. arXiv preprint arXiv:2410.00362.
    [CrossRef] [Google Scholar]
  36. Fan, T., Ma, G., Kang, Y., Gu, H., Song, Y., Fan, L., ... & Yang, Q. (2025, January). Fedmkt: Federated mutual knowledge transfer for large and small language models. In Proceedings of the 31st International Conference on Computational Linguistics (pp. 243-255). https://aclanthology.org/2025.coling-main.17/
    [Google Scholar]
  37. Lin, Z., Hu, X., Zhang, Y., Chen, Z., Fang, Z., Chen, X., ... & Gao, Y. (2024). Splitlora: A split parameter-efficient fine-tuning framework for large language models. arXiv preprint arXiv:2407.00952.
    [CrossRef] [Google Scholar]
  38. Lin, Z., Zhang, Y., Chen, Z., Fang, Z., Chen, X., Vepakomma, P., Ni, W., Luo, J., & Gao, Y. (2026). HSplitLoRA: A heterogeneous split parameter-efficient fine-tuning framework for large language models. IEEE Transactions on Mobile Computing, 25(10), 15543-15559.
    [CrossRef] [Google Scholar]
  39. Chen, X., Li, L., Ji, F., & Wu, W. (2025, May). Memory-efficient split federated learning for llm fine-tuning on heterogeneous mobile devices. In IEEE INFOCOM 2025-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) (pp. 1-6). IEEE.
    [CrossRef] [Google Scholar]
  40. Zhao, K., Yang, Z., Hu, Y., Chen, M., Zhu, C., & Zhang, Z. (2025). Efficient Split Federated Learning for Large Language Models over Communication Networks. arXiv preprint arXiv:2504.14667.
    [CrossRef] [Google Scholar]
  41. Shan, Y., Zhang, Z., Di, S., Liu, Y., Lu, X., & Liu, B. (2026). SplitFT: An Adaptive Federated Split Learning System For LLMs Fine-Tuning. arXiv preprint arXiv:2604.26388.
    [CrossRef] [Google Scholar]
  42. Geng, J., Yuan, T., Han, P., Gao, Y., Chen, X., & Luo, B. (2025). FlexP-SFT: A flexible aggregation-free framework for on-device personalized split federated fine-tuning of LLMs. arXiv preprint arXiv:2508.10349.
    [CrossRef] [Google Scholar]
  43. Kou, Z., Chen, Z., Yang, J., & Shen, C. (2026). Lean Clients, Full Accuracy: Hybrid Zeroth-and First-Order Split Federated Learning. arXiv preprint arXiv:2601.09076.
    [CrossRef] [Google Scholar]
  44. Geng, J., Chen, X., & Luo, B. (2026). Just Talk Once: Communication-Efficient Split Federated LLM Fine-Tuning on Edge Devices. arXiv preprint arXiv:2609.01457.
    [CrossRef] [Google Scholar]
  45. Yang, X., Li, L., Li, S., Guan, L., Wang, H., Qi, X., ... & Pan, M. (2025). Fed MobiLLM: Efficient Federated LLM Fine-Tuning over Heterogeneous Mobile Devices via Server Assisted Side-Tuning. arXiv preprint arXiv:2508.06765.
    [CrossRef] [Google Scholar]

Cite This Article

APA Style
Shen, Y., Liu, X., Yang, Y., & Lv, H. (2026). Federated Fine-Tuning of Large Language Models on Resource-Constrained Clients: Technical Approaches, Resource Costs, and Applicability. Journal of Reliable and Secure Computing, 2(3), 194-211. https://doi.org/10.62762/JRSC.2026.225854
Export Citation
RIS Format
Compatible with EndNote, Zotero, Mendeley, and other reference managers
TY  - JOUR
AU  - Shen, Yunheng
AU  - Liu, Xiao
AU  - Yang, Yang
AU  - Lv, Hairong
PY  - 2026
DA  - 2026/09/20
TI  - Federated Fine-Tuning of Large Language Models on Resource-Constrained Clients: Technical Approaches, Resource Costs, and Applicability
JO  - Journal of Reliable and Secure Computing
T2  - Journal of Reliable and Secure Computing
JF  - Journal of Reliable and Secure Computing
VL  - 2
IS  - 3
SP  - 194
EP  - 211
DO  - 10.62762/JRSC.2026.225854
UR  - https://www.icck.org/article/abs/JRSC.2026.225854
KW  - federated learning
KW  - large language models
KW  - privacy-preserving computation
KW  - trustworthy AI
KW  - parameter-efficient fine-tuning
KW  - zeroth-order optimization
KW  - proxy models
KW  - split learning
KW  - secure aggregation
KW  - resource accounting
AB  - Federated fine-tuning adapts language models to distributed data and is widely adopted as a privacy-preserving alternative to centralized training, yet constrained clients must still store model weights and training states, execute updates, and communicate with a server. This review examines four composable routes---parameter-efficient and quantized adaptation, backpropagation-free adaptation, proxy or submodel adaptation, and split federated adaptation---through a common framework that traces the objects each endpoint retains, exchanges, and discloses. When a client retains the complete base, reducing adapter state leaves a base-storage floor; boundary communication depends on input dimensions, sequence length, and interaction frequency, so parameter count alone is insufficient. Keeping data local constrains the raw dataset but not the transmitted object: adapter updates, boundary activations, logits, and cached embeddings expose different surfaces, and the encryption, secure-aggregation, or noise mechanisms that protect them add to the same budgets. Server weight sharing removes duplication but retains private states and caches, and combining routes requires recalculating these costs for the resulting protocol. The analysis identifies where savings arise, where costs move, what each route discloses, and which training, deployment, and trustworthiness conditions govern its applicability.
SN  - 3070-6424
PB  - Institute of Central Computation and Knowledge
LA  - English
ER  -
BibTeX Format
Compatible with LaTeX, BibTeX, and other reference managers
@article{Shen2026Federated,
  author = {Yunheng Shen and Xiao Liu and Yang Yang and Hairong Lv},
  title = {Federated Fine-Tuning of Large Language Models on Resource-Constrained Clients: Technical Approaches, Resource Costs, and Applicability},
  journal = {Journal of Reliable and Secure Computing},
  year = {2026},
  volume = {2},
  number = {3},
  pages = {194-211},
  doi = {10.62762/JRSC.2026.225854},
  url = {https://www.icck.org/article/abs/JRSC.2026.225854},
  abstract = {Federated fine-tuning adapts language models to distributed data and is widely adopted as a privacy-preserving alternative to centralized training, yet constrained clients must still store model weights and training states, execute updates, and communicate with a server. This review examines four composable routes---parameter-efficient and quantized adaptation, backpropagation-free adaptation, proxy or submodel adaptation, and split federated adaptation---through a common framework that traces the objects each endpoint retains, exchanges, and discloses. When a client retains the complete base, reducing adapter state leaves a base-storage floor; boundary communication depends on input dimensions, sequence length, and interaction frequency, so parameter count alone is insufficient. Keeping data local constrains the raw dataset but not the transmitted object: adapter updates, boundary activations, logits, and cached embeddings expose different surfaces, and the encryption, secure-aggregation, or noise mechanisms that protect them add to the same budgets. Server weight sharing removes duplication but retains private states and caches, and combining routes requires recalculating these costs for the resulting protocol. The analysis identifies where savings arise, where costs move, what each route discloses, and which training, deployment, and trustworthiness conditions govern its applicability.},
  keywords = {federated learning, large language models, privacy-preserving computation, trustworthy AI, parameter-efficient fine-tuning, zeroth-order optimization, proxy models, split learning, secure aggregation, resource accounting},
  issn = {3070-6424},
  publisher = {Institute of Central Computation and Knowledge}
}

Article Metrics

Citations
Crossref
0
Scopus
0
Views
22
PDF Downloads
5

Publisher's Note

ICCK stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and Permissions

CC BY Copyright © 2026 by the Author(s). Published by Institute of Central Computation and Knowledge. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
Journal of Reliable and Secure Computing
Journal of Reliable and Secure Computing
ISSN: 3070-6424 (Online)
Portico
Preserved at
Portico

Article QR Code

Article QR Code
Scan to read this article
Submit Manuscript Edit a Special Issue

Popular Articles

Share This Article