Skip to main content
Have a personal or library account? Click to login
Applying a Community-based Participatory Research Approach to the Development of a Structured Assessment Tool: A Case Study in a Multiservice Youth Initiative Cover

Applying a Community-based Participatory Research Approach to the Development of a Structured Assessment Tool: A Case Study in a Multiservice Youth Initiative

Open Access
|Nov 2018

Figures & Tables

Table 1

Results from a content analysis of provider interview data.

ThemesCodesNetwork Agencies
(n = 3)
Case Management
(n = 5)
Street Outreach Team
(n = 1)
Total Sample
(n = 9)
Providers lack a shared understanding about how to use the toolTool administered differentially based on provider capacity and personal preferences3519
Differences in administration style vary by staff member and youth characteristics2417
Lack of uniformity for information collection procedures3407
Calls to referral source (i.e., parents/schools) to obtain more information if necessary3205
Assessment completed after decision-making/case planning2204
Helpful when one designated person completes all initial risk assessments1102
Uncertainty exists around the level of training received by all providers3215
Providers found some items difficult to administerItem and domain helpfulness/usefulness varies by agency as well as staff roles3519
Translating items from risk assessment into youth-friendly language can be difficult3317
Providers do not use the tool to answer culturally sensitive questions0101
Tool has the potential to be useful if item-level revisions are made3519
The tool needs a clear purpose and relevance to case planningTool does not serve a clear purpose in the initiative3519
Tool not considered useful for case planning2215
Providers use information from the referral form to determine youth needs3407
Referral form is most practical way for determining youth needs3306
Qualitative section on referral form most helpful for assessing youth needs0213
Information collection practices to develop service plans vary widely3519
Tool does not guide decision-making but has potential to be effective and applicable3418
Assessment of risk/need should be a well-defined and supported process3418
Develop a scoring system will improve usability3115
Tool format and assessment procedures should be compatible to the service settingTimeframe for completing initial assessment is too short3418
Uncertainty about the reliability of information collected with the tool because of time constraints and limited rapport building before assessments3317
Tool is too long and wordy3306
Tool has the potential to be useful if ordering of items is reformatted1102
Tool does not target the goals and missions of different agencies1102
Table 2

Overview of project activities and partner roles during phase.

Project StepCBPR PrincipleAcademic Researcher RoleCommunity Provider RoleJoint Decision Making
Step 1: Convene a Collaborative Workgroup
  • Disseminate results to all partners

  • Facilitate a collaborative, equitable partnership in all phases of the research

  • Build on strengths and resources within the community

  • Balance research and on-the-ground application

  • Provide overview of qualitative interview findings and recommendations for building a collaborative partnership

  • Discuss benefits of academic-community partnership and co-design process for promoting tool ownership and sustainability

  • Facilitate discussion of partner roles and workgroup goals

  • Coordinate workgroup scheduling

  • Provide suggestions for frequency of meetings and workgroup goals

  • Describe preferences for provider roles and responsibilities in revision process

  • Elicit perspectives from other (non-workgroup) initiative providers and present this information to the workgroup

  • Decide collaborative workgroup is an appropriate strategy for tool revision process

  • Agree on partner roles and responsibilities

  • Identify key purpose of structured assessment across initiative components

Step 2: Tool Adaptation and Assessment Protocol Standardization
  • Facilitate a collaborative, equitable partnership in all phases of the research

  • Build on strengths and resources within the community

  • Foster co-learning and capacity building

  • Balance research and on-the-ground application

  • Involve systems development in a cyclical and iterative process

  • Facilitate discussion of domain and item-level revisions

  • Synthesize provider feedback and make initial item-level revisions accordingly

  • Translate results from youth focus groups into initial youth self-report tool draft

  • Conduct systematic literature review to develop item weighting

  • Develop and beta test tool scoring procedures; work with initiative database vendor to develop a graphical representation of assessment data to inform case planning

  • Provide tool revision suggestions, including adding/removing items, wording, format, and structure

  • Provide feedback and suggestions on how to best implement the tool across diverse agencies/provider roles

  • Co-facilitate youth focus groups to support development of self-report short form

  • Provide feedback on scoring system and challenges with data management system

  • Review practice protocols, provide feedback and revision suggestions

  • Decide on questions, wording, format, and structure of final tool

  • Engage in iterative tool revision process, including development of youth self-report form

  • Agree on standardized practice protocols for tool use

  • Agree on scoring strategy and use of data

Step 3: Quality Assurance and Tool Re-Implementation
  • Facilitate a collaborative, equitable partnership in all phases of the research

  • Builds on strengths and resources within the community

  • Integrate co-learning and capacity building

  • Disseminate results to all partners and involve them in wider dissemination

  • Review evidence-based principles of structured assessment and components of quality assurance (QA) systems

  • Discuss common challenges that may affect systematic implementation efforts

  • Develop training on new scoring and use of data in real-time

  • Hire and work with motivational interviewing (MI) trainer to develop new tool training

  • Develop reports and wider dissemination products; Write initial resource drafts

  • Provide feedback about the type of QA activities that are realistic for service settings

  • Provide formal input during hiring process for QA manager

  • Provide feedback on QA and dissemination products

  • Develop case studies for MI training that are based on professional experience and knowledge

  • Agree on key quality assurance components

  • Hire a full-time quality assurance manager

  • Co-author technical manual, training webinar, assessment protocol resources

  • Develop and attend all-staff training on MI and revised assessment tool/practices

[i] Note: Partner roles did not often align with any single CBPR principle; as such, roles are not listed in order of principle.

Table 3

Missing data item analysis for original and adapted assessment tools.

Assessment tool domains & subdomainsMissing data n (%)
Original tool
(n = 174)
Adapted tool
(n = 454)
% Differencec
Academic status4 (2%)12 (3%)–1%
Attendance15 (9%)16 (4%)5%
Suspension/expulsion14 (9%)14 (3%)6%
Peer12 (7%)35 (8%)–1%
Romantic, intimate or sexual18 (11%)12 (3%)8%
Gang affiliation12 (7%)43 (9%)–2%
Use of Free Time6 (3%)7 (2%)1%
Employment11 (6%)4 (1%)5%
Aggression17 (10%)18 (4%)6%
Criminal History23 (13%)26 (6%)7%
Criminal statusa113 (25%)
Alcohol use30 (18%)17 (4%)14%
Historical alcohol usea18 (4%)
Drug use17 (10%)16 (4%)6%
Historical drug usea22 (5%)
Family issues10 (6%)9 (2%)4%
Domestic violence44 (26%)19 (4%)22%
Housing status9 (5%)13 (3%)2%
Teen parenting and pregnancyb161 (93%)21 (5%)
Family member incarcerationb120 (69%)28 (6%)
Towards anti-social behavior26 (15%)23 (5%)10%
Towards fighting and physical aggression to resolve conflict18 (11%)9 (2%)9%
Towards school47 (27%)4 (1%)26%
Mental health issues30 (18%)25 (6%)12%
Support network13 (7%)13 (3%)4%
Trauma44 (25%)29 (6%)19%

[i] Notes: Percentages rounded to the nearest hundredth. Bolded text indicates domains; non-bolded text indicates subdomains. aAdapted tool separated historical and current use into separate subdomains. bOriginal tool only provided a check box for “yes” responses – no distinction could be made between missing data and an endorsement of “not applicable.” cA negative value indicates the adapted tool had higher missingness.

DOI: https://doi.org/10.33596/coll.19 | Journal eISSN: 2638-4396
Language: English
Page range: 5 - 5
Published on: Nov 22, 2018
Published by: University of Miami Libraries
In partnership with: Paradigm Publishing Services

© 2018 Asia S. Bishop, Sarah C. Walker, Mariko Lockhart, Todd I. Herrenkohl, published by University of Miami Libraries
This work is licensed under the Creative Commons Attribution 4.0 License.