PADS: Physics-Aware, Model-Free Diffusion Training for Robust Generative Intelligence in Optical Neural Networks
Research Article  ·  Published: 20 November 2026
Issue cover
Optical Wireless Communication
Volume 1, Issue 1, 2026: 60-75
Research Article Open Access

PADS: Physics-Aware, Model-Free Diffusion Training for Robust Generative Intelligence in Optical Neural Networks

1 Shangrao Normal University, Shangrao 334001, Jiangxi, China
* Corresponding Author: Qibao Huang, [email protected]
Volume 1, Issue 1
You have full access to this open access article · CC BY 4.0 License

Article Information

Pages 60-75

Abstract

Integrating diffusion models into physical computing systems such as optical neural networks (ONNs) enables low-power, high-speed generative intelligence but faces two critical barriers: the simulation-to-reality (sim2real) gap and the prohibitive computational cost of gradient-based training with differentiable simulators. The iterative denoising of diffusion models (typically 1000 steps) accumulates errors from physical imperfections (aberrations, misalignments, quantization), causing severe performance degradation when the models are deployed on real hardware. To address this, we propose PADS (Physics-Aware Diffusion Training via Score Gradient Estimation), a model-free optimization framework that removes simulator dependency by formulating physical parameter learning as a black-box optimization problem and optimizing the parameters directly with Monte Carlo score function estimation. Validated on a diffractive optical system with realistic non-idealities, PADS achieves FID scores of 38.7 on Fashion-MNIST (FMNIST) and 74.3 on CIFAR-10, outperforming simulator-based training (SBT) by 46% and enhanced hybrid training (HBT+) by 19% on CIFAR-10. It also reduces GPU memory consumption by 99% (from 14.8 GB to 0.2 GB), which makes edge deployment feasible. This work advances physical generative intelligence by modeling compound sim2real errors, developing a zeroth-order, model-free training framework, and providing experimental validation on a diffractive optical system.

Graphical Abstract

PADS: Physics-Aware, Model-Free Diffusion Training for Robust Generative Intelligence in Optical Neural Networks

Keywords

diffusion models physical neural networks model-free optimization zeroth-order optimization optical computing

Data Availability Statement

The data generated during the current study are available from the corresponding author upon reasonable request. Fashion-MNIST is publicly available at https://github.com/zalandoresearch/fashion-mnist and CIFAR-10 is publicly available at https://www.cs.toronto.edu/~%7B%7Dkriz/cifar.html.

Funding

This work was supported without any funding.

Conflicts of Interest

The authors declare that they have no competing interests.

AI Use Statement

The authors declare that no generative AI was used in the preparation of this manuscript.

Ethical Approval and Consent to Participate

Not applicable. This study did not involve human participants or animals.

References

  1. Kazerouni, A., Aghdam, E. K., Heidari, M., Azad, R., Fayyaz, M., Hacihaliloglu, I., & Merhof, D. (2023). Diffusion models in medical imaging: A comprehensive survey. Medical Image Analysis, 88, 102846.
    [CrossRef] [Google Scholar]
  2. Kim, S. W., Philion, J., Torralba, A., & Fidler, S. (2021, June). DriveGAN: Towards a controllable high-quality neural simulation. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 5816-5825). IEEE.
    [CrossRef] [Google Scholar]
  3. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022, June). High-resolution image synthesis with latent diffusion models. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 10674-10685). IEEE.
    [CrossRef] [Google Scholar]
  4. Bond-Taylor, S., Leach, A., Long, Y., & Willcocks, C. G. (2022). Deep generative modelling: A comparative review of VAEs, GANs, normalizing flows, energy-based and autoregressive models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11), 7327-7347.
    [CrossRef] [Google Scholar]
  5. Papamakarios, G., Nalisnick, E., Rezende, D. J., Mohamed, S., & Lakshminarayanan, B. (2021). Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57), 1-64. https://www.jmlr.org/papers/v22/19-1028.html
    [Google Scholar]
  6. Ho, J., Jain, A., & Abbeel, P. (2020). Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33, 6840-6851.
    [Google Scholar]
  7. Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2020). Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456.
    [CrossRef] [Google Scholar]
  8. Wright, L. G., Onodera, T., Stein, M. M., Wang, T., Schachter, D. T., Hu, Z., & McMahon, P. L. (2022). Deep physical neural networks trained with backpropagation. Nature, 601(7894), 549-555.
    [CrossRef] [Google Scholar]
  9. Lin, X., Rivenson, Y., Yardimci, N. T., Veli, M., Luo, Y., Jarrahi, M., & Ozcan, A. (2018). All-optical machine learning using diffractive deep neural networks. Science, 361(6406), 1004-1008.
    [CrossRef] [Google Scholar]
  10. Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017, September). Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 23-30). IEEE.
    [CrossRef] [Google Scholar]
  11. Xue, Z., Zhou, T., Xu, Z., Yu, S., Dai, Q., & Fang, L. (2024). Fully forward mode training for optical neural networks. Nature, 632(8024), 280-286.
    [CrossRef] [Google Scholar]
  12. Zhao, G., Shu, X., & Zhou, R. (2025). High-performance real-world optical computing trained by in situ gradient-based model-free optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(9), 7194-7205.
    [CrossRef] [Google Scholar]
  13. Song, J., Meng, C., & Ermon, S. (2020). Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502.
    [CrossRef] [Google Scholar]
  14. Xiao, Z., Kreis, K., & Vahdat, A. (2021). Tackling the generative learning trilemma with denoising diffusion GANs. arXiv preprint arXiv:2112.07804.
    [CrossRef] [Google Scholar]
  15. Oguz, I., Dinc, N. U., Yildirim, M., Ke, J., Yoo, I., Wang, Q., ... & Psaltis, D. (2024). Optical diffusion models for image generation. Advances in Neural Information Processing Systems, 37, 59150-59173.
    [CrossRef] [Google Scholar]
  16. Chen, S., Li, Y., Wang, Y., Chen, H., & Ozcan, A. (2025). Optical generative models. Nature, 644(8078), 903-911.
    [CrossRef] [Google Scholar]
  17. Wang, Y., Zhong, M., Liao, K., Sun, Q., Hu, X., & Gong, Q. (2026). Diffractive optical neural networks: Pathways toward intelligent and scalable photonic computing. Nanophotonics, 15(8), e70098.
    [CrossRef] [Google Scholar]
  18. Williams, R. J. (1992). Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3-4), 229-256.
    [CrossRef] [Google Scholar]
  19. Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., ... & Kavukcuoglu, K. (2016, June). Asynchronous methods for deep reinforcement learning. In International Conference on Machine Learning (pp. 1928-1937). PMLR. https://proceedings.mlr.press/v48/mniha16.html
    [Google Scholar]
  20. Wang, Z., Zoghi, M., Hutter, F., Matheson, D., & de Freitas, N. (2016). Bayesian optimization in a billion dimensions via random embeddings. Journal of Artificial Intelligence Research, 55, 361-387.
    [CrossRef] [Google Scholar]
  21. Mohamed, S., Rosca, M., Figurnov, M., & Mnih, A. (2020). Monte Carlo gradient estimation in machine learning. Journal of Machine Learning Research, 21(132), 1-62. https://www.jmlr.org/papers/v21/19-346.html
    [Google Scholar]
  22. Jang, E., Gu, S., & Poole, B. (2016). Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144.
    [CrossRef] [Google Scholar]
  23. Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
    [CrossRef] [Google Scholar]
  24. Matsushima, K., & Shimobaba, T. (2009). Band-limited angular spectrum method for numerical simulation of free-space propagation in far and near fields. Optics Express, 17(22), 19662-19673.
    [CrossRef] [Google Scholar]
  25. Xiao, H., Rasul, K., & Vollgraf, R. (2017). Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747.
    [CrossRef] [Google Scholar]
  26. Krizhevsky, A. (2009). Learning multiple layers of features from tiny images (Technical report). University of Toronto. Retrieved from https://cave.cs.toronto.edu/kriz/learning-features-2009-TR.pdf
    [Google Scholar]
  27. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017). GANs trained by a two time-scale update rule converge to a local Nash equilibrium. Advances in Neural Information Processing Systems, 30, 6626-6637.
    [Google Scholar]
  28. Kynkäänniemi, T., Karras, T., Laine, S., Lehtinen, J., & Aila, T. (2019). Improved precision and recall metric for assessing generative models. Advances in Neural Information Processing Systems, 32.
    [Google Scholar]
  29. Nichol, A. Q., & Dhariwal, P. (2021, July). Improved denoising diffusion probabilistic models. In International Conference on Machine Learning (pp. 8162-8171). PMLR. https://proceedings.mlr.press/v139/nichol21a.html
    [Google Scholar]
  30. Karras, T., Aittala, M., Hellsten, J., Laine, S., Lehtinen, J., & Aila, T. (2020). Training generative adversarial networks with limited data. Advances in Neural Information Processing Systems, 33, 12104-12114.
    [Google Scholar]
  31. Karras, T., Aittala, M., Aila, T., & Laine, S. (2022). Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems, 35, 26565-26577.
    [CrossRef] [Google Scholar]
  32. Brühl, E., Buckup, T., & Motzkus, M. (2017). Minimization of $1/f^n$ phase noise in liquid crystal masks for reliable femtosecond pulse shaping. Optics Express, 25(19), 23376-23386.
    [CrossRef] [Google Scholar]
  33. Matsumoto, N., Itoh, H., Inoue, T., Otsu, T., & Toyoda, H. (2014). Stable and flexible multiple spot pattern generation using LCOS spatial light modulator. Optics Express, 22(20), 24722.
    [CrossRef] [Google Scholar]
  34. Salimans, T., & Ho, J. (2022). Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.00512.
    [CrossRef] [Google Scholar]
  35. Zhou, T., Lin, X., Wu, J., Chen, Y., Xie, H., Li, Y., Fan, J., Wu, H., Fang, L., & Dai, Q. (2021). Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nature Photonics, 15(5), 367-373.
    [CrossRef] [Google Scholar]

Cite This Article

APA Style
Huang, Q., Zhao, A., & Xu, X. (2026). PADS: Physics-Aware, Model-Free Diffusion Training for Robust Generative Intelligence in Optical Neural Networks. Optical Wireless Communication, 1(1), 60-75. https://doi.org/10.62762/OWC.2026.824441
Export Citation
RIS Format
Compatible with EndNote, Zotero, Mendeley, and other reference managers
TY  - JOUR
AU  - Huang, Qibao
AU  - Zhao, Ai'an
AU  - Xu, Xinye
PY  - 2026
DA  - 2026/11/20
TI  - PADS: Physics-Aware, Model-Free Diffusion Training for Robust Generative Intelligence in Optical Neural Networks
JO  - Optical Wireless Communication
T2  - Optical Wireless Communication
JF  - Optical Wireless Communication
VL  - 1
IS  - 1
SP  - 60
EP  - 75
DO  - 10.62762/OWC.2026.824441
UR  - https://www.icck.org/article/abs/owc.2026.824441
KW  - diffusion models
KW  - physical neural networks
KW  - model-free optimization
KW  - zeroth-order optimization
KW  - optical computing
AB  - Integrating diffusion models into physical computing systems such as optical neural networks (ONNs) enables low-power, high-speed generative intelligence but faces two critical barriers: the simulation-to-reality (sim2real) gap and the prohibitive computational cost of gradient-based training with differentiable simulators. The iterative denoising of diffusion models (typically 1000 steps) accumulates errors from physical imperfections (aberrations, misalignments, quantization), causing severe performance degradation when the models are deployed on real hardware. To address this, we propose PADS (Physics-Aware Diffusion Training via Score Gradient Estimation), a model-free optimization framework that removes simulator dependency by formulating physical parameter learning as a black-box optimization problem and optimizing the parameters directly with Monte Carlo score function estimation. Validated on a diffractive optical system with realistic non-idealities, PADS achieves FID scores of 38.7 on Fashion-MNIST (FMNIST) and 74.3 on CIFAR-10, outperforming simulator-based training (SBT) by 46% and enhanced hybrid training (HBT+) by 19% on CIFAR-10. It also reduces GPU memory consumption by 99% (from 14.8 GB to 0.2 GB), which makes edge deployment feasible. This work advances physical generative intelligence by modeling compound sim2real errors, developing a zeroth-order, model-free training framework, and providing experimental validation on a diffractive optical system.
SN  - 5 Articles Required
PB  - Institute of Central Computation and Knowledge
LA  - English
ER  - 
BibTeX Format
Compatible with LaTeX, BibTeX, and other reference managers
@article{Huang2026PADS,
  author = {Qibao Huang and Ai'an Zhao and Xinye Xu},
  title = {PADS: Physics-Aware, Model-Free Diffusion Training for Robust Generative Intelligence in Optical Neural Networks},
  journal = {Optical Wireless Communication},
  year = {2026},
  volume = {1},
  number = {1},
  pages = {60-75},
  doi = {10.62762/OWC.2026.824441},
  url = {https://www.icck.org/article/abs/owc.2026.824441},
  abstract = {Integrating diffusion models into physical computing systems such as optical neural networks (ONNs) enables low-power, high-speed generative intelligence but faces two critical barriers: the simulation-to-reality (sim2real) gap and the prohibitive computational cost of gradient-based training with differentiable simulators. The iterative denoising of diffusion models (typically 1000 steps) accumulates errors from physical imperfections (aberrations, misalignments, quantization), causing severe performance degradation when the models are deployed on real hardware. To address this, we propose PADS (Physics-Aware Diffusion Training via Score Gradient Estimation), a model-free optimization framework that removes simulator dependency by formulating physical parameter learning as a black-box optimization problem and optimizing the parameters directly with Monte Carlo score function estimation. Validated on a diffractive optical system with realistic non-idealities, PADS achieves FID scores of 38.7 on Fashion-MNIST (FMNIST) and 74.3 on CIFAR-10, outperforming simulator-based training (SBT) by 46\% and enhanced hybrid training (HBT+) by 19\% on CIFAR-10. It also reduces GPU memory consumption by 99\% (from 14.8 GB to 0.2 GB), which makes edge deployment feasible. This work advances physical generative intelligence by modeling compound sim2real errors, developing a zeroth-order, model-free training framework, and providing experimental validation on a diffractive optical system.},
  keywords = {diffusion models, physical neural networks, model-free optimization, zeroth-order optimization, optical computing},
  issn = {5 Articles Required},
  publisher = {Institute of Central Computation and Knowledge}
}

Article Metrics

Citations
Crossref
0
Scopus
0
Views
3
PDF Downloads
0

Publisher's Note

ICCK stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and Permissions

CC BY Copyright © 2026 by the Author(s). Published by Institute of Central Computation and Knowledge. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
Optical Wireless Communication
Optical Wireless Communication
ISSN: 5 Articles Required (Online)
Portico
Preserved at
Portico