Accuracy is the closeness of any assessment to its actual value. Validity and reliability are 2 important aspects of any measurement for accuracy. Validity refers to accurately measuring what it claims to measure while reliability is an assessment of reproducibility and consistency of a measurement or instrument. For any instrument to be valid it should measure what it intends to measure. E.g.:- If an instrument is used to measure caries, it should measure caries and not the enamel hyperplasia.

If a machine weighs 40 Kg instead of 50 Kg weight, the validity of instrument is questionable (Table 1). But if instrument measures 40 Kg repeatedly it is a reliable instrument.

Types of Validity

External Validity

External validity is about generalization. If a study produces similar results among different kinds of populations, the result is said to be generalized. If the population behaves differently (lack of population validity) or environment influences behaviour (lack of ecological validity) the result cannot be generalized. Study done at high altitude may be different from that done at low altitude. Similarly, maturational time of adolescents may be different for different countries for various reasons.

Internal validity

Internal validity ensures good experimental design to produce accurate results. Appropriate construct and content of the questionnaire, randomization, blinding, matching ensure sound methodological concept for concrete results. Results need to be interpreted in an objective and critical way, before assessing their implications and drawing conclusions [1].

Types of Internal Validity

• Construct

- Convergent

- Discriminant

• Content validity
• Criteria validity.
• Face validity (Logical validity) and Apparent validity

Face validity (Logical validity) and apparent validity

Face validity should be judged by subjective assessment and relevance of the questionnaire to the participants, face validity is determined by a review of the items and not through the use of statistical analysis.

Face validity is a measure of how representative a research project is ‘at face value’ and either appears to be a good project.

For example: if we have to collect the sample of handicapping malocclusion and if there are cases with normal occlusion and mild malocclusion are also included in the study we can visualize and say that normal occlusion and mild malocclusion are not fit for the study.

Similarly, if we have to collect the sample of aggressive periodontitis and there comes a case of marginal gingivitis or periodontitis, we can easily differentiate between them for face validity.

Whereas in Apparent validity, question is generated in consultation with opinion of experts and subjects themselves. All questions should be logical.

Construct Validity

It consists of demonstrating that the measure is related to other similar measures of the similar characteristics and not related to other characteristics. It defines how well a test or experiment measures up-to its claim and degree to which instrument reflects the concepts, how well the instrument is constructed [2].

Convergent Validity

Convergent validity was conducted by correlating scores between Dental Fear Scale (DFS), total score and global question (do you fear going to dental office). Convergent validity is a subtype of construct validity, it determines whether 2 similar constructs corresponds with each other or not while discriminant validity shows differentiation between 2 dissimilar groups that can be easily differentiated. Convergent validity shows high correlation whereas discriminant validity shows low correlation coefficient.

Convergent validity of Dental Fear Scale (DFS) was calculated by correlating scores between global question and Dental Fear Scale (DFS) (Table 2).

Discriminant Validity

This type of validity tests concepts that have no relationship to each other.

For example:

Content Validit

Most dimensions of concept are included to evaluate its material/advice is sought from experts, pilot study/factorial analysis. Content validity ratio refers to extent to which a measure represents all facets of a given construct. An element of subjectivity to determine content validity, in which a degree of agreement is requires about what a particular personality trait such as extraversion represents. The gain of a high content validity prevented by a disagreement about a personality trait.

For example:

1. In order to assess the content validity of Caries Assessment Spectrum And Treatment (CAST) Index in Indian population, twenty Subject Matter Experts (five each from Conservative, Paediatrics, Public health and Oral medicine) were asked to determine if each of codes (0-9) of cast index were essential or non essential and also justify the reason for it by writing their comments for each code [3] (Table 4).

2. Formula for Content Validity Ratio (CVR),

   CVR= (n_e-N/2) / (N/2)

   n_e - no of dentist with post-graduate degree qualification indicating “essential”.

   N - Total no. of dentist with post-graduate degree qualification.

   (0.5 and above - Acceptable; 0.4 and below - Not acceptable; CVR score 1 is highly acceptable). On the basis of CVR score code 1, 4, 9 were found to be non essential for Indian population.

3. Criteria Validity

4. It is a best measure of validity.It assess whether a test reflects certain set of abilities like concurrent validity and predictive validity.

5. Concurrent validity

6. It compares result of two things (new and old test) at same time. Comparing newly designed test with already existing gold standard at the same time for concurrent validity to check whether new test is valid or not.

7. Predictive validity

8. It is a measure of how well a test predicts. It involves testing, subjects and comparing them at some point in future. For example, if a student appearing in medical entrance excels we can predict that he will perform better in future also.

    

9. Validation through ROC (Receivers Operating Curve) / Sensitivity and Specificity
10. Receiver Operating Characteristic (ROC)

11. It is a graphical plot that represents performance of a binary classifier system as its discrimination threshold is varied. This curve plots the True Positive Rate (TPR) against the False Positive Rate (FPR). The true-positive rate is also known as sensitivity, recall or probability of detection whereas false-positive rate is also known as the probability of false alarm or fall-out and can be calculated as (1 -specificity) [5].

12. Receiver operating characteristic analysis provides tools which select possibly optimal models. From the cost context or the class distribution it also discards suboptimal ones independently (and prior to specifying). In a direct and natural way this analysis is related to cost/benefit analysis of diagnostic decision making [5].

     

13. Sensitivity

14. Sensitivity measures the proportion of positives that are correctly identified (e.g., the condition which correctly identifies the percentage of sick people who are having disease) [5].

15. Specificity

16. Specificity measures the proportion of negatives that are correctly identified as such (e.g., the measurement of healthy people who are correctly identified as not having the condition) [5].

    Example: To assess the sagittal relationship between maxilla and mandible with accuracy and reproducibility a new cephalometric measurement introduced, named the W angle [6].

17. Receiver Operating Characteristic Curve use to examine the sensitivity and specificity of W angle to discriminate between 3 different skeletal pattern group (Class I, Class II, Class III).ROC showed that W angle less than 51⁰ has 96% sensitivity and 90% specificity for discriminating the class II group from class I group. A w angle greater than 56⁰ has 95% sensitivity and 98% specificity for discriminating the class III group from class I group .

18. Therefore ROC shows that;

    Cut-off points

    Between class I & class II is 51⁰

    Between class I & class III is 56⁰

19. Two versions of Community periodontal Index and Treatment Need (CPITN) partial and full version were validated using ROC Curves in Brazilian population [7].
20. Reliability does not depend on validity while validity depends on reliability also. It should weigh accurately to be a valid instrument. Methodology of any research should be full proof to give accurate result. Any kind of bias in the study will distort the findings. If selection bias is not prevented by allocation concealment /randomization, while allocating patients to different study groups the result will be altered. Similarly, if there is no blinding and the investigator / analyser know about the groups he is measuring there may be bias affecting the final result. Thus method and methodology both should be valid to produce accurate result. This will reduce systemic error.

21. Accuracy is lack of errors. For any result to be accurate it should be free from any kind of bias and at the same time instrument / method should be a gold standard method or a valid method. Similarly random error should be minimised to increase the precision. Precision is opposite to random error. Thus both validity and reliability are important aspect of accuracy. Systemic errors are caused by different forms of bias affecting the mean value. Random errors do not affect the mean value but affect reliability.

    Validity and reliability are threatened by different forms of bias [8]. For example - Selection bias, recall bias and non-response bias etc.

     For equal distribution of confounding factors there should be perfect matching between experimental and control group. Proper selection method (Randomization) should be followed to allocate patients to each group. The method of allocation should not be known to the researcher (Allocation concealment). Blinding of subject, investigator and analyser helps in controlling different forms of bias. E.g., Behaviour bias. Appropriate steps should be taken to avoid recall bias, non response bias etc. All these precautions ensure sound methodology (Table6).

22. There is a proper technique to achieve validity and reliability [11].To perform any research study often we have to create valid and reliable questionnaire / test. There are certain guidelines to carefully construct and develop its contents so that most dimensions of concept are included by seeking advice from experts [11-13]. It should be well constructed to reflect the concept. Its results should correlate with gold standard method. High Positive correlation shows good convergent validity. Similarly discriminate validity shows low correlation between 2 different groups with theoretically different concepts.A question developed in a different country may not be a valid measure in our country. Just because questionnaire has been piloted or used in previous studies does not mean it is valid or reliable. Questionnaire should be cross culturally adopted and validated.

23. Conclusion

24. Qualities of any research depends on various factors like validity of instrument, intra and inter observer reliability, presence of known and unknown confounding factors, presence of bias etcTo enhance quality of the study, researchers should take precautions in designing studies to reduce bias, which can be achieved through perfect matching, blinding and adapting appropriate selection method.

     

    Questionnaire should have proper content validity and should adapt according to the need of study. Sound and concrete results can only be produced with appropriate method and methodology.

25.  

     

1. Jain S, Jain S, Punyani PR, Jain D (2015) Basics of interpreting results. International Dental & Medical Journal of Advanced Research 1: 1-4.
2. Saunders B, Trapp R (1998) Basics and Clinical Biostatistics. (2nd edition), United states of America: Prentice-Hall International Inc, USA. Pg No: 58-59.
3. Phansopkar S, Hegde-Shetiya S, Devadiga A, Agrawal D, Mahuli A, et al. (2015) Face and Content Validation of Caries Assessment Spectrum and Treatment Index among Few Subject Matter Experts in India. International Journal of Dental Health Concerns 1: 13-18.
4. Jain S, Sharma N, Jain D (2015) Basic fundaments of designing a quality research. Journal of Advanced Medical and Dental Sciences Research 3: 88-95.
5. En.wikipedia.org (2017) Receiver operating characteristic.
6. Bhad W, Nayak S, Doshi U (2011) A new approach of assessing sagittal dysplasia: the W angle. The European Journal of Orthodontics 35: 66-70.
7. Bassani D, Silva C, Oppermann R (2006) Validity of the Community Periodontal Index of Treatment Needs' (CPITN) for population periodontitis screening. Cadernos de Saúde Pública 22: 277-283.
8. Jain S, Debbarma S, Jain D (2016) Bias in Dental Research / Dentistry. Annals of International Medical and Dental Research 2: 2395-2814.
9. Fleming PS, Johal A (2010) Self-ligating brackets in orthodontics. A systematic review. The Angle Orthodontics 80: 575-584.
10. Jain S, Sharma N (2016) Guideline for systematic reviews. International Dental and Medical Journal of Advance Research 2: 1-10.
11. Jain S, Dubey S, Jain S (2016) Designing and validation of a questionnaire. International dental & medical journal of advanced research 2: 1-3.
12. Edwards P (2010) Questionnaires in clinical trials: guidelines for optimal design and administration. Trials 11.
13. Kazi A, Khalid W (2012) Questionnaire designing and validation. Journal of Pakistan Medical Associatio 62: 514-516.