## Principles of Diagnostic Yields

Now that we have examined how to get examples of Endoscopic performance by analysis of documentation, we can perform the more interesting task of determining endoscopic performance by determining the pathology taken at endoscopy. This means we have to merge two datasets and then extract pathology based on the endoscopy results.

We will determine the dysplasia detection rate in our sample of Barrett’s endoscopies and we will do this per endoscopist so we can get an overview of who is the best at detecting dysplasia in Barrett’s oesopahgus. Note this is very similar to the concept of an adenoma detection rate in colonoscopy.

We start by defining our datasets. We have done this before as part of the Surveillance page so I’m not going to repeat it here. We can just start using it.

This is what the merged dataset looks like (truncated for ease of viewing):

EndoHistoMerge<-source('EndoPathMerged_ExternalCode.R')
EndoHistoMerge<-data.frame(EndoHistoMerge)
#Neaten up the names
names(EndoHistoMerge)<-gsub("value.","",names(EndoHistoMerge),fixed=T)
EndoHospNumId EndoReports Date.x Endoscopist Midazolam Fentanyl Indication Diagnosis BarrC BarrM Histop_dfHospNumId HistoReport Date.y Macro Diagnoses Days visible
P433224 Date of Procedure 2015-09-21 Endoscopist: Dr Bilbo Baggins Midazolam: 3mg Fentanyl: 12.5mcg Indication: Haematemesis or Melaena/Blood PR Diagnosis: Food bolus obstructing the oesophagus..Esophageal candidiasis .Oesophagitis. .Barretts oesophagus. .Possible achalasia. Barrett’s oesophagus length: C0M5 2015-09-21 Bilbo Baggins 3 12.5 Haematemesis or Melaena/Blood PR Diagnosis: Food bolus obstructing the oesophagus..Esophageal candidiasis .Oesophagitis. .Barretts oesophagus. .Possible achalasia. Barrett’s oesophagus length: C0M5 0 5 P433224 Date received: 2015-09-23 Macrosopic description: 7 specimens collected the largest measuring 5 x 3 x 3 mm and the smallest 5 x 5 x 3 mm Diagnoses The appearances are those of Candida oesophagitis..The appearances are consistent with the endoscopic diagnosis of Barrett’s oesophagus with active chronic inflammation..There is some ulceration. 2015-09-23 7 specimens collected the largest measuring 5 x 3 x 3 mm and the smallest 5 x 5 x 3 mm The appearances are those of Candida oesophagitis..The appearances are consistent with the endoscopic diagnosis of Barrett’s oesophagus with active chronic inflammation..There is some ulceration. 2 days FALSE
P433224 Date of Procedure 2015-09-21 Endoscopist: Dr Bilbo Baggins Midazolam: 3mg Fentanyl: 12.5mcg Indication: Haematemesis or Melaena/Blood PR Diagnosis: Food bolus obstructing the oesophagus..Esophageal candidiasis .Oesophagitis. .Barretts oesophagus. .Possible achalasia. Barrett’s oesophagus length: C0M5 2015-09-21 Bilbo Baggins 3 12.5 Haematemesis or Melaena/Blood PR Diagnosis: Food bolus obstructing the oesophagus..Esophageal candidiasis .Oesophagitis. .Barretts oesophagus. .Possible achalasia. Barrett’s oesophagus length: C0M5 0 5 P433224 Date received: 2015-09-23 Macrosopic description: 2 specimens collected the largest measuring 4 x 2 x 4 mm and the smallest 2 x 4 x 5 mm Diagnoses There is low grade dysplasia.There is no significant increase in intraepithelial eosinophils..The biopsies of oesophageal squamous mucosa show surface erosion and active chronic inflammation..Numerous Candida spores and hyphae are present admixed with ulcer slough.. No herpetic viral inclusions are seen..The appearances are consistent with the endoscopic diagnosis of Barrett’s oesophagus with active chronic inflammation. 2015-09-23 2 specimens collected the largest measuring 4 x 2 x 4 mm and the smallest 2 x 4 x 5 mm There is low grade dysplasia.There is no significant increase in intraepithelial eosinophils..The biopsies of oesophageal squamous mucosa show surface erosion and active chronic inflammation..Numerous Candida spores and hyphae are present admixed with ulcer slough.. No herpetic viral inclusions are seen..The appearances are consistent with the endoscopic diagnosis of Barrett’s oesophagus with active chronic inflammation. 2 days FALSE
P433224 Date of Procedure 2015-09-21 Endoscopist: Dr Bilbo Baggins Midazolam: 3mg Fentanyl: 12.5mcg Indication: Haematemesis or Melaena/Blood PR Diagnosis: Food bolus obstructing the oesophagus..Esophageal candidiasis .Oesophagitis. .Barretts oesophagus. .Possible achalasia. Barrett’s oesophagus length: C0M5 2015-09-21 Bilbo Baggins 3 12.5 Haematemesis or Melaena/Blood PR Diagnosis: Food bolus obstructing the oesophagus..Esophageal candidiasis .Oesophagitis. .Barretts oesophagus. .Possible achalasia. Barrett’s oesophagus length: C0M5 0 5 P433224 Date received: 2015-09-20 Macrosopic description: 3 specimens collected the largest measuring 3 x 5 x 4 mm and the smallest 2 x 1 x 5 mm Diagnoses There is low grade dysplasia.The appearances are consistent with, but not specific for Barrett’s (columnar lined) oesophagus..High grade dysplasia is present throughout this sample.Intestinal metaplasia is present..The appearances are consistent with the endoscopic diagnosis of Barrett’s oesophagus with active chronic inflammation.. No herpetic viral inclusions are seen..No Helicobacter are seen. 2015-09-20 3 specimens collected the largest measuring 3 x 5 x 4 mm and the smallest 2 x 1 x 5 mm There is low grade dysplasia.The appearances are consistent with, but not specific for Barrett’s (columnar lined) oesophagus..High grade dysplasia is present throughout this sample.Intestinal metaplasia is present..The appearances are consistent with the endoscopic diagnosis of Barrett’s oesophagus with active chronic inflammation.. No herpetic viral inclusions are seen..No Helicobacter are seen. 1 days FALSE
U873352 Date of Procedure 2017-01-10 Endoscopist: Dr Florence Nightingale Midazolam: 2mg Fentanyl: 12.5mcg Indication: Oesophagus- Dysplasia Diagnosis: Gastritis.Extensive neoplastic looking esophageal lesion. .Food bolus obstructing the oesophagus..Esophageal candidiasis .Post chemo-radiotherapy stricture .Hiatus Hernia. .Barretts oesophagus. .Ulcer- Oesophageal. .Oesophagitis. .Possible achalasia. Barrett’s oesophagus length: C3M7 2017-01-10 Florence Nightingale 2 12.5 Oesophagus- Dysplasia Diagnosis: Gastritis.Extensive neoplastic looking esophageal lesion. .Food bolus obstructing the oesophagus..Esophageal candidiasis .Post chemo-radiotherapy stricture .Hiatus Hernia. .Barretts oesophagus. .Ulcer- Oesophageal. .Oesophagitis. .Possible achalasia. Barrett’s oesophagus length: C3M7 3 7 U873352 Date received: 2017-01-10 Macrosopic description: 1 specimens collected the largest measuring 5 x 2 x 3 mm and the smallest 3 x 1 x 1 mm Diagnoses There is low grade dysplasia.There is no significant increase in intraepithelial eosinophils..The appearances are consistent with the endoscopic diagnosis of Barrett’s oesophagus with active chronic inflammation..The appearances are consistent with, but not specific for Barrett’s (columnar lined) oesophagus..No granulomas or viral inclusions are seen..There is some ulceration. 2017-01-10 1 specimens collected the largest measuring 5 x 2 x 3 mm and the smallest 3 x 1 x 1 mm There is low grade dysplasia.There is no significant increase in intraepithelial eosinophils..The appearances are consistent with the endoscopic diagnosis of Barrett’s oesophagus with active chronic inflammation..The appearances are consistent with, but not specific for Barrett’s (columnar lined) oesophagus..No granulomas or viral inclusions are seen..There is some ulceration. 0 days FALSE
U873352 Date of Procedure 2017-01-10 Endoscopist: Dr Florence Nightingale Midazolam: 2mg Fentanyl: 12.5mcg Indication: Oesophagus- Dysplasia Diagnosis: Gastritis.Extensive neoplastic looking esophageal lesion. .Food bolus obstructing the oesophagus..Esophageal candidiasis .Post chemo-radiotherapy stricture .Hiatus Hernia. .Barretts oesophagus. .Ulcer- Oesophageal. .Oesophagitis. .Possible achalasia. Barrett’s oesophagus length: C3M7 2017-01-10 Florence Nightingale 2 12.5 Oesophagus- Dysplasia Diagnosis: Gastritis.Extensive neoplastic looking esophageal lesion. .Food bolus obstructing the oesophagus..Esophageal candidiasis .Post chemo-radiotherapy stricture .Hiatus Hernia. .Barretts oesophagus. .Ulcer- Oesophageal. .Oesophagitis. .Possible achalasia. Barrett’s oesophagus length: C3M7 3 7 U873352 Date received: 2017-01-09 Macrosopic description: 1 specimens collected the largest measuring 1 x 2 x 1 mm and the smallest 3 x 1 x 4 mm Diagnoses The appearances are those of Candida oesophagitis.. No herpetic viral inclusions are seen..The appearances are consistent with the endoscopic diagnosis of Barrett’s oesophagus with active chronic inflammation..There is no dysplasia or malignancy. 2017-01-09 1 specimens collected the largest measuring 1 x 2 x 1 mm and the smallest 3 x 1 x 4 mm The appearances are those of Candida oesophagitis.. No herpetic viral inclusions are seen..The appearances are consistent with the endoscopic diagnosis of Barrett’s oesophagus with active chronic inflammation..There is no dysplasia or malignancy. 1 days FALSE

# Breakdown of analytical method for yield

So the task now involves extracting the presence of dysplasia in those patients where Barrett’s was detected at endoscopy and then grouping them by endoscopist

So to spell out where each element of this comes from:

1. Detect rows that mention Barrett’s in the endoscopy report.

2. Also detect rows that mention dysplasia in the pathology report.

3. Group by endoscopist.

4. Then get the total number of reports that mention Barrett’s in the endoscopy report whether dyplasia is metioned or not, and group by endoscopist.

5. Calculate proportion by endoscopist then visualise it.

### 1 & 2. Detect rows that mention term of interest in the endoscopy report.

This is done using grepl. We will do a combined grepl so we can get the subset we are interested in and that mention dysplasia in the pathology report.

DysplasticBarretts<-EndoHistoMerge[grepl("[Bb]arrett",EndoHistoMerge$EndoReports)&grepl("[Dd]ysplasi",EndoHistoMerge$HistoReport),]

So you will note that there are several reports where it is mentioned that there is no dysplasia so we have to get rid of these. This is tricky but for the purposes of this site we will use the brute force technique

DysplasticBarretts<-DysplasticBarretts[!grepl("[Nn]either dysplasia",DysplasticBarretts$Diagnoses)&!grepl("[Nn]o [Dd]ysplasia",DysplasticBarretts$Diagnoses),]

### 3. Group by endoscopist

There are two ways of doing this. The first way is to use dplyr as follows:

EndoscopistDDRBarretts<-DysplasticBarretts%>%group_by(Endoscopist)%>%summarise(n=n())

kable(EndoscopistDDRBarretts)
Endoscopist n
Bilbo Baggins 105
Bugs Bunny 108
Charles Dickens 88
Chubby Checker 99
Davy Jones 96
Elmo Fudd 124
Elvis Presley 100
Florence Nightingale 93
Frank Sinatra 94
Jimminey Cricket 103
Jonny Begood 100
King Richard III 110
Rara Rasputin 107

Alternatively we can just use the table function which has a neater input but gives a messier output:

DDRtable<-table(EndoscopistDDRBarretts)
kable(DDRtable)
88 93 94 96 99 100 103 105 107 108 110 124
Bilbo Baggins 0 0 0 0 0 0 0 1 0 0 0 0
Bugs Bunny 0 0 0 0 0 0 0 0 0 1 0 0
Charles Dickens 1 0 0 0 0 0 0 0 0 0 0 0
Chubby Checker 0 0 0 0 1 0 0 0 0 0 0 0
Davy Jones 0 0 0 1 0 0 0 0 0 0 0 0
Elmo Fudd 0 0 0 0 0 0 0 0 0 0 0 1
Elvis Presley 0 0 0 0 0 1 0 0 0 0 0 0
Florence Nightingale 0 1 0 0 0 0 0 0 0 0 0 0
Frank Sinatra 0 0 1 0 0 0 0 0 0 0 0 0
Jimminey Cricket 0 0 0 0 0 0 1 0 0 0 0 0
Jonny Begood 0 0 0 0 0 1 0 0 0 0 0 0
Joseph Conrad 1 0 0 0 0 0 0 0 0 0 0 0
King Richard III 0 0 0 0 0 0 0 0 0 0 1 0
Rara Rasputin 0 0 0 0 0 0 0 0 1 0 0 0
Sal Addin 0 0 0 0 1 0 0 0 0 0 0 0

### 4. Get the total number of specific endoscopies by endoscopist:

So now we know who is picking up dysplasia we can express this as a proportion of all the Barrett’s endoscopy they have done as follows:

AllBarretts<-EndoHistoMerge[grepl("[Bb]arrett",EndoHistoMerge$EndoReports),] Endoscopist_All_Barretts<-AllBarretts%>%group_by(Endoscopist)%>%summarise(n=n()) ### 5. DDR by endoscopist: Now we just calulate the proportions to get the DDR. We have to bind Endoscopist_All_Barretts and EndoscopistDDRBarretts to calculate this. We merge by endoscopist #For a bit of variety we are going to do the merge using dplyr join functions instead of base R merge functions: DDRTable<-full_join(Endoscopist_All_Barretts, EndoscopistDDRBarretts, by = "Endoscopist") And finally the proportions: DDRTable$Prop<-(DDRTable$n.y/DDRTable$n.x)*100
DDRTable<-data.frame(DDRTable)
#Lets get rid of NA values by replacing with "0"
DDRTable$Prop[is.na(DDRTable$Prop)] <- 0

#Lets plot it out
barplot(DDRTable$Prop,names.arg=DDRTable$Endoscopist, ylab = "% dysplasia",
cex.lab = 1.5,cex.axis=1.0,cex.main = 1.0,cex.names=1.0,main = "Barrett's Endoscopic Dysplasia detection proportion",las=2)

….Looks like some people need more training…