Data Insights: Validation, Reliability, and Healthcare Statistics
Turning Data into Useful Information
Visualization – understand the data, discover hidden patterns for data analysis and storytelling.
Descriptive statistics – story about what happened, an overview of the state of the situation, etc.
Baseline performance, Performance Indicators – e.g., Dashboard.
Detect trends, identify patterns, and uncover relationships.
What is Data Validation?
Data validation is the process of ensuring that a program operates on clean, correct, and useful data. It uses routines, often called “validation rules”, that check for correctness, meaningfulness, and security of data that are input to the system.
Reliability and Consistency in Measurement
Reliability is consistency in measurement.
Repeatability: Reduce the variation in repeated measurement results taken by the same person/instrument on the same record, and under the same condition.
Reproducibility: Reduce the variation in repeated measurement results taken by different persons on the same records and under the same condition.
Valid but not reliable: Valid because the pattern is evenly distributed around the correct goal: the person tried to hit the correct place. Not reliable because the shots are off the mark in every possible direction. They are not consistent.
Reliable but not valid: Reliable: Pattern shows shot hits the same part of the target each time. It is consistent, so reliable. Not valid: The goal is to hit the center of the target, but shots are not in the area.
Neither Reliable nor valid: Not Reliable because the shots are not tightly clustered together, so not consistent. Not Valid: because to the extent there is any pattern, it is not at the true target, the center.
Both Reliable and Valid: Reliable: the darts land close together. The player can hit the same part of the target. Valid: The darts are clustered at the center where they are aimed.
Why are Validity and Reliability Important?
The measurement is only useful and has meaning if it measures what it is supposed to measure (valid) and does so with accuracy and consistency (reliability).
Who Performs Data Validation?
An independent person or team not involved in the original data collection.
Types of Data Validation
- Entry level: It ensures that all the fields marked as mandatory are filled correctly before clicking the submit button, e.g., check digits check (NRIC), format check (DD/MM/YY), etc.
- Field level: It makes certain to enter correct value across specific data field, e.g., character, format, limit, logic, range check, and conformance test.
- Cardinality: Checks that record has a valid related records, e.g., if payroll database is marked as “resigned”, then this record must not have any associated salary payment after the date on which staff left the organization, etc.
- Batch: Checks for missing records, e.g., mortality figures can be validated by comparing inpatient discharge records and death reported by mortuary, etc.
- Field and Cross System Consistency: Checks related fields (e.g., FV and ConRV ?, etc.) – compare data in different systems (e.g., First_Name, Middle_Name, and Last_Name – patient’s name with the same ID is the same in all the computer systems, etc.)
- Inter-Rater Reliability: Assesses the consistency of how a measuring system is implemented.
Data Validation Methods
- Measurement Results agreement
- Data Element agreement
Sample Size in Data Validation
Statistical precision – Confidence Level (CL) and Creditability – Confidence Interval (CI)
When to do Data Validation
[a] A new measure is implemented (b) Data will be made public on the organization’s website or in other ways (c) A change has been made to an existing measure, such as the data collection tools have changed or the data abstraction process or abstractor has changed (d) The data resulting from an existing measure have changed in an unexplainable way (e) The data source has changed, such as when part of the patient record has been turned into an electronic format and thus the data source is now both electronic and paper; or (f) The subject of the data collection has changed, such as changes in average of patients, comorbidities, research protocol alterations, new practice guidelines implemented, or new technologies and treatment methodologies introduced.
Statistics in Healthcare
1. Vital Statistics 2. Inpatient 3. Intensive Care Unit (ICU) 4. Health Facility 5. Accident & Emergency (A&E) 6. Outpatient (Clinic and Ambulatory) 7. Disease Burden 8. Healthcare Cost 9. Health Manpower
Vital Statistics
1. Fertility 2. Birth 3. Death/ Mortality 4. Life Expectancy
Fertility Rate
(No. of Live Births / No. of women between 15 to 44 years) X 1000
Birth Rate
(No. of Births /No. of Population) X 1,000
Crude Neonatal Mortality Rate
(No. of Neonatal Deaths /No. of Live Births) X1000
Infant Mortality Rate
(No. of Children died <= 1 year of age / No. of Live Births) X 1000
Maternal Mortality Rate
(No. of maternal deaths due to birth or pregnancy-related complications/ No. of Live Births) X 1000000
Death Rate
(No. of Death/ No. of Population) X 1000
Inpatient Statistics
Admissions Newborn admissions
Bed Complement Available beds (Beds in service) Bed days Inpatients Patient days (Length of Stay) Average Length of Stay (ALOS)
Transfer
Discharge (Separation) Death (mortality)
Care should be more than 24 hours
Bed Occupancy Turnover rate Turnover interval
Falls Readmission
Bed Complement Vs Bed in Service
Beds in Service
Average daily census of available beds.
An available bed is one that is immediately available to be used by an admitted patient or resident if required.
Bed Occupancy Rate (BOR)
(Total Patient Days) /{(No. of Days in the reporting period) X (Beds in Service)} X 100
(Total Patient Days)/{(No. of Days in the reporting period) X (Beds in Service)} X 100
Inpatient Mortality and Observed Inpatient Mortality
(Number of Inpatient Deaths/Number of Inpatient Discharges) X 100
Perioperative Mortality
(Number of Perioperative deaths for all ASA classes/Number of inpatient anesthesia episodes for all ASA classes)X 100
Reported Inpatient Falls Per 1,000 Patient Days
(Number of reported falls in inpatient wards/Number of Patient Days) X 1000
Readmissions
(No. of Emergency Readmissions <= 30 days/ No. of Inpatient Discharges) X 100
Exclusions: • Death cases • Discharge against advice • Cases transferred to other hospitals • Conditions susceptible to frequent readmissions (e.g. Cancer, etc,)