Medical Image Report Generation Method Integrating Cross-View Features and Attention Mechanisms

China Medical Devices ›› 0

China Medical Devices ›› 0 DOI: 10.3969/j.issn.1674-1633.20241568
RESEARCH WORK

Medical Image Report Generation Method Integrating Cross-View Features and Attention Mechanisms

Author information +
History +

Abstract

Objective To explore a strategy for aligning multi-view medical images with medical reports to optimize the quality of automatically generated medical imaging reports by deep learning models. Methods A generation model integrating multi-view features and attention mechanisms was designed. Firstly, it encodes the features of medical images taken from different angles and the pathological description features of reports based on a pre-trained model. Then, various attention mechanisms are utilized to complete the fusion calculation of the features. Finally, the decoder is used to translate the combined features into case reports. Results After multiple rounds of testing on the authoritative and publicly available chest IU X-Ray datasets, the model was on average 8.34%, 14.2%, 10.9%, 6.14%, 1.7%, and 5.5% higher than the previously proposed methods in six typical evaluation metrics, with a comprehensive performance improvement of 7.79%. Conclusion The model performs well in terms of the accuracy and fluency of the generated reports, validating that the integration strategy effectively captures the underlying relationships between images and reports, thereby enhancing the model's report generation capabilities.

Key words

Medical Imaging; Encoder-Decoder; Multi-View Features; Attention Mechanism; Automatic Report Generation

Cite this article

Download Citations
Medical Image Report Generation Method Integrating Cross-View Features and Attention Mechanisms[J]. China Medical Devices, 0 https://doi.org/10.3969/j.issn.1674-1633.20241568

36

Accesses

0

Citation

Detail

Sections
Recommended

/