Medical Image Report Generation Method Integrating Cross-View Features and Attention Mechanisms
Author information+
{{custom_zuoZheDiZhi}}
{{custom_authorNodes}}
{{custom_bio.content}}
{{custom_bio.content}}
{{custom_authorNodes}}
Show less
History+
Just Accepted Date
2025-03-11
Abstract
Objective To explore a strategy for aligning multi-view medical images with medical reports to optimize the quality of automatically generated medical imaging reports by deep learning models. Methods A generation model integrating multi-view features and attention mechanisms was designed. Firstly, it encodes the features of medical images taken from different angles and the pathological description features of reports based on a pre-trained model. Then, various attention mechanisms are utilized to complete the fusion calculation of the features. Finally, the decoder is used to translate the combined features into case reports. Results After multiple rounds of testing on the authoritative and publicly available chest IU X-Ray datasets, the model was on average 8.34%, 14.2%, 10.9%, 6.14%, 1.7%, and 5.5% higher than the previously proposed methods in six typical evaluation metrics, with a comprehensive performance improvement of 7.79%. Conclusion The model performs well in terms of the accuracy and fluency of the generated reports, validating that the integration strategy effectively captures the underlying relationships between images and reports, thereby enhancing the model's report generation capabilities.
Medical Image Report Generation Method Integrating Cross-View Features and Attention Mechanisms[J]. China Medical Devices, 0 https://doi.org/10.3969/j.issn.1674-1633.20241568