Content Validity and Cognitive Testing in the Development of a Motivational Interviewing Self-Assessment Questionnaire

Tadeja HOČEVAR; Tim ANSTISS; Danica ROTAR PAVLIČ

doi:10.2478/sjph-2024-0007

.blurhash-client-img { display: none !important; }

Content Validity and Cognitive Testing in the Development of a Motivational Interviewing Self-Assessment Questionnaire

Slovenian Journal of Public Health

Volume 63 (2024): Issue 1 (March 2024)

By: Tadeja HOČEVAR , Tim ANSTISS and Danica ROTAR PAVLIČ

Open Access

|Jan 2024

Figures & Tables

Stages and steps in testing the content validity of the MI practice self-assessment questionnaire.
Legend: [] – numbers in brackets define steps in testing content validity;MIQ 1.0/2.=/3.0 – version of MI questionnaire

Table 1.

The content validity of the measurement instrument as a whole and by specific MI elements, with the universal agreement of experts (S-CVI-UA).

MI scale/element	S-CVI-UA¹
MI scale/element	RELEVANCE of the question	understandability of the question	COMPLETENESS of response options	MEANINGFULNESS of response options
Scale as a whole	0.93	0.67	0.93	0.85
Partnership	1.00	0.33	1.00	0.67
Acceptance	0.80	0.40	0.80	0.80
Evoking	0.75	1.00	1.00	1.00
Resisting the righting reflex	1.00	0.67	0.83	0.83
Strengthening self-efficacy	1.00	1.00	1.00	1.00

1 Legend:

11 S-CVI-UA = the proportion of the items the experts scored as valid (ratings 3 or 4); cut-off point: S-CVI-UA≥0.80 (10)

Table 2.

The content validity of the measurement instrument as a whole and by specific MI elements, with the universal agreement of experts (S-CVI-UA).

MI scale/element	S-CVI-Ave¹
MI scale/element	RELEVANCE of the question	UNDERSTANDABILITY of the question	COMPLETENESS of response options	MEANINGFULNESS of response options
Scale as a whole	0.99	0.93	0.98	0.98
Partnership	1.00	0.86	0.94	0.94
Acceptance	0.97	0.73	0.97	0.97
Evoking	0.96	1.00	1.00	1.00
Resisting the righting reflex	1.00	0.94	0.95	0.97
Strengthening self-efficacy	1.00	0.95	1.00	1.00

1 Legend:

11 S-CVI-Ave = the average proportion of the items rated 3 or 4; cut-off score: S-CVI-Ave≥0.90 (10)

Table 3.

Values of the validity index for individual items (I-CVI) and the modified kappa coefficient (k^*) for 27 items.

MI element	Item code^* and content	RELEVANCE of the question				UNDERSTANDABILITY of the question				COMPLETENESS of response options				MEANINGFULNESS of response options
MI element	Item code^* and content	N¹_(exp3-4)	I-CVI²	P_c³	k^{*, 4}	N¹_(exp3-4)	I-CVI²	P_c³	k^{*, 4}	N¹_(exp3-4)	I-CVI²	P_c³	k^{*, 4}	N¹_(exp3-4)	I-CVI²	P_c³	k^{*, 4}
PARTNERSHIP	P1 make P/C⁵ feel comfortable	6	1.00	0.000	1.00	5	0.83	0.094	0.67	5	0.83	0.094	0.67	5	0.83	0.094	0.67
	P2 being supportive	6	1.00	0.000	1.00	5	0.83	0.094	0.67	5	0.83	0.094	0.67	5	0.83	0.094	0.67
	P3 working together	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	P4 P/C’s input	6	1.00	0.000	1.00	4	0.67	0.234	0.33	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	P5 incorporate P/C’s ideas	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	P6 be there in case P/C changes their mind	6	1.00	0.000	1.00	5	0.83	0.094	0.67	6	1.00	0.000	1.00	6	1.00	0.000	1.00
ACCEPTANCE	A1 P/C’s view is relevant	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	A2 strive to understand	6	1.00	0.000	1.00	5	0.83	0.094	0.67	5	0.83	0.094	0.67	5	0.83	0.094	0.67
	A3 P/C’s choice to change	6	1.00	0.s000	1.00	5	0.83	0.094	0.67	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	A4 respect P/C’s decision	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	A5 P/C’s personal growth	5	0.83	0.094	0.67	4	0.67	0.234	0.33	6	1.00	0.000	1.00	6	1.00	0.000	1.00
EVOKING	E2 P/C’s own reasons	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	E3 P/C’s own strategies	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	E4 encourage P/C’s thinking	5	0.83	0.094	0.67	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	E5 P/C’s inner strenghts and sources	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00
RESISTING THE RIGHTING REFLEX	R1^* explaining without first exploring	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	R2^* talking about own knowing	6	1.00	0.000	1.00	5	0.83	0.094	0.67	5	0.83	0.094	0.67	5	0.83	0.094	0.67
	R3^* reasons without permission and inquire	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	R4^* ideas without permission and inquire	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	R5^* talk P/C into	6	1.00	0.000	1.00	5	0.83	0.094	0.67	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	R6 suggestions after permission and inquire	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00
STRENGTHENING SELF-EFFICACY	S1 ask about confidence	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	S2 ask about needed	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	S3 P/C’s past experiences	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	S5 affirmations	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	S6 change talk	6	1.00	0.000	1.00	5	0.83	0.094	0.67	6	1.00	0.000	1.00	6	1.00	0.000	1.00
	S8 other resources	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00	6	1.00	0.000	1.00

1 Legend:

1* reverse scaling

11 N(exp3-4) = number of experts providing a rating of 3 or 4

12 I-CVI (content validity index) = number of experts providing a rating of 3 or 4/number of experts; cut-off score: I-CVI≥0.78 (9); automatic item rejection: I-CVI<0.50 (11)

13 pc (probability of chance occurence) = [N!/A!(N-A)!] x 0,5N N = number of experts; A = number of experts agreeing on a rating of 3 or 4 (11)

14 k* (modified kappa) = (I-CVI-pc)(1-pc)

15 P/C = patient/client

References

Authors

Metrics

Articles in this issue

DOI: https://doi.org/10.2478/sjph-2024-0007 | Journal eISSN: 1854-2476 | Journal ISSN: 0351-0026

Journal RSS Feed

Language: English, Slovenian

Page range: 46 - 54

Submitted on: Oct 5, 2023

Accepted on: Nov 28, 2023

Published on: Jan 3, 2024

Published by: National Institute of Public Health, Slovenia

In partnership with: Paradigm Publishing Services

Keywords:

Alcohol screening and brief intervention,

Cognitive testing,

Content validity,

Motivational interviewing,

Self-assessment questionnaire

Related subjects:

Medicine,

Clinical medicine,

Hygiene and environmental medicine

© 2024 Tadeja HOČEVAR, Tim ANSTISS, Danica ROTAR PAVLIČ, published by National Institute of Public Health, Slovenia
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Volume 63 (2024): Issue 1 (March 2024)

Content Validity and Cognitive Testing in the Development of a Motivational Interviewing Self-Assessment Questionnaire

Figures & Tables

Figure 1.

Table 1.

Table 2.

Table 3.

Paradigm

My account