Osteoarthritis of the knee (KOA) is one of the main causes of disability; hence, it requires precise and early evaluation of the severity level of the disease. In this paper, we propose a multimodal deep learning architecture based on a self-supervised Swin transformer combined with a cross-modal attention mechanism (SWIN-MULTI-ATTEN) for combining radiological images and patients' information (age, sex, and BMI). This framework captures both structural and contextual information to achieve better classification accuracy. Experimental results on the Osteoarthritis Initiative (OAI) dataset show that the proposed model achieves an accuracy of 91.4% and a QWK of 0.903, which outperforms other C... More >