anicolson commited on
Commit
6f29ffd
·
verified ·
1 Parent(s): 677dccc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +56 -56
README.md CHANGED
@@ -1,26 +1,26 @@
1
- ---
2
- language:
3
- - en
4
- license: apache-2.0
5
- library_name: transformers
6
- tags:
7
- - chest X-ray report generation
8
- - radiology report generation
9
- - image captioning
10
- - chest X-ray
11
- - X-ray
12
- - radiology
13
- - cxrmate
14
- - cxrmate-ed
15
- - report
16
- - radiology report
17
- - multimodal
18
- - patient data
19
- - patient records
20
- - mimic-cxr
21
- - mimic-iv-ed
22
- pipeline_tag: image-text-to-text
23
- ---
24
 
25
  # **CXRMate-ED**: The Impact of Auxiliary Patient Data on Automated Chest X-Ray Report Generation and How to Incorporate It
26
 
@@ -30,39 +30,6 @@ The abstract from the paper:
30
 
31
  "This study investigates the integration of diverse patient data sources into multimodal language models for automated chest X-ray (CXR) report generation. Traditionally, CXR report generation relies solely on CXR images and limited radiology data, overlooking valuable information from patient health records, particularly from emergency departments. Utilising the MIMIC-CXR and MIMIC-IV-ED datasets, we incorporate detailed patient information such as vital signs, medicines, and clinical history to enhance diagnostic accuracy. We introduce a novel approach to transform these heterogeneous data sources into embeddings that prompt a multimodal language model; this significantly enhances the diagnostic accuracy of generated radiology reports. Our comprehensive evaluation demonstrates the benefits of using a broader set of patient data, underscoring the potential for enhanced diagnostic capabilities and better patient outcomes through the integration of multimodal data in CXR report generation."
32
 
33
- ## MIMIC-CXR & MIMIC-IV-ED Dataset:
34
-
35
- MIMIC-CXR, MIMIC-CXR-JPG, and MIMIC-IV-ED must be in the same Physio Net directory. E.g.:
36
-
37
- ```shell
38
- user@cluster:~$ ls /home/user/physionet.org/files
39
- mimic-cxr mimic-cxr-jpg mimic-iv-ed
40
- ```
41
-
42
- ### Download MIMIC-CXR-JPG:
43
-
44
- Download the MIMIC-CXR-JPG dataset from https://physionet.org/content/mimic-cxr-jpg, e.g.,
45
- ```shell
46
- wget -r -N -c -np --user <username> --ask-password https://physionet.org/files/mimic-cxr-jpg/2.1.0/
47
- ```
48
- Note that you must be a credentialised user to access this dataset.
49
-
50
- ### Download the reports from MIMIC-CXR:
51
-
52
- MIMIC-CXR-JPG does not include the radiology reports and are instead included with MIMIC-CXR (the DICOM version of the dataset). To download this dataset and avoid downloading the DICOM files (which are very large), use `--reject dcm` with the wget command from https://physionet.org/content/mimic-cxr, e.g,
53
- ```shell
54
- wget -r -N -c -np --reject dcm --user <username> --ask-password https://physionet.org/files/mimic-cxr/2.0.0/
55
- ```
56
- Note that you must be a credentialised user to access this dataset.
57
-
58
- ### Download MIMIC-IV-ED:
59
-
60
- Download the MIMIC-IV-ED dataset from https://physionet.org/content/mimic-iv-ed, e.g.,
61
- ```shell
62
- wget -r -N -c -np --user <username> --ask-password https://physionet.org/files/mimic-iv-ed/2.2/
63
- ```
64
- Note that you must be a credentialised user to access this dataset.
65
-
66
  ### Prepare the dataset:
67
 
68
  ```python
@@ -123,6 +90,39 @@ for i,j in zip(findings, impression):
123
  print(f'Findings:\t{i}\nImpression:\t{j}\n\n')
124
  ```
125
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
126
  # Environment requirements
127
 
128
  Environment requirements can be found here: https://github.com/aehrc/cxrmate-ed/blob/main/requirements.txt.
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: apache-2.0
5
+ library_name: transformers
6
+ tags:
7
+ - chest X-ray report generation
8
+ - radiology report generation
9
+ - image captioning
10
+ - chest X-ray
11
+ - X-ray
12
+ - radiology
13
+ - cxrmate
14
+ - cxrmate-ed
15
+ - report
16
+ - radiology report
17
+ - multimodal
18
+ - patient data
19
+ - patient records
20
+ - mimic-cxr
21
+ - mimic-iv-ed
22
+ pipeline_tag: image-text-to-text
23
+ ---
24
 
25
  # **CXRMate-ED**: The Impact of Auxiliary Patient Data on Automated Chest X-Ray Report Generation and How to Incorporate It
26
 
 
30
 
31
  "This study investigates the integration of diverse patient data sources into multimodal language models for automated chest X-ray (CXR) report generation. Traditionally, CXR report generation relies solely on CXR images and limited radiology data, overlooking valuable information from patient health records, particularly from emergency departments. Utilising the MIMIC-CXR and MIMIC-IV-ED datasets, we incorporate detailed patient information such as vital signs, medicines, and clinical history to enhance diagnostic accuracy. We introduce a novel approach to transform these heterogeneous data sources into embeddings that prompt a multimodal language model; this significantly enhances the diagnostic accuracy of generated radiology reports. Our comprehensive evaluation demonstrates the benefits of using a broader set of patient data, underscoring the potential for enhanced diagnostic capabilities and better patient outcomes through the integration of multimodal data in CXR report generation."
32
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  ### Prepare the dataset:
34
 
35
  ```python
 
90
  print(f'Findings:\t{i}\nImpression:\t{j}\n\n')
91
  ```
92
 
93
+ ## MIMIC-CXR & MIMIC-IV-ED Dataset:
94
+
95
+ MIMIC-CXR, MIMIC-CXR-JPG, and MIMIC-IV-ED must be in the same Physio Net directory. E.g.:
96
+
97
+ ```shell
98
+ user@cluster:~$ ls /home/user/physionet.org/files
99
+ mimic-cxr mimic-cxr-jpg mimic-iv-ed
100
+ ```
101
+
102
+ ### Download MIMIC-CXR-JPG:
103
+
104
+ Download the MIMIC-CXR-JPG dataset from https://physionet.org/content/mimic-cxr-jpg, e.g.,
105
+ ```shell
106
+ wget -r -N -c -np --user <username> --ask-password https://physionet.org/files/mimic-cxr-jpg/2.1.0/
107
+ ```
108
+ Note that you must be a credentialised user to access this dataset.
109
+
110
+ ### Download the reports from MIMIC-CXR:
111
+
112
+ MIMIC-CXR-JPG does not include the radiology reports and are instead included with MIMIC-CXR (the DICOM version of the dataset). To download this dataset and avoid downloading the DICOM files (which are very large), use `--reject dcm` with the wget command from https://physionet.org/content/mimic-cxr, e.g,
113
+ ```shell
114
+ wget -r -N -c -np --reject dcm --user <username> --ask-password https://physionet.org/files/mimic-cxr/2.0.0/
115
+ ```
116
+ Note that you must be a credentialised user to access this dataset.
117
+
118
+ ### Download MIMIC-IV-ED:
119
+
120
+ Download the MIMIC-IV-ED dataset from https://physionet.org/content/mimic-iv-ed, e.g.,
121
+ ```shell
122
+ wget -r -N -c -np --user <username> --ask-password https://physionet.org/files/mimic-iv-ed/2.2/
123
+ ```
124
+ Note that you must be a credentialised user to access this dataset.
125
+
126
  # Environment requirements
127
 
128
  Environment requirements can be found here: https://github.com/aehrc/cxrmate-ed/blob/main/requirements.txt.