1 Appropriateness: Does the instrument or item measure what the members of the Steering Committee thought was appropriate, given the stated purposes of the GFA and the project?
2 Reliability: Does the instrument measure the concept in a consistent and stable way? The instruments selected for inclusion should display good inter-rater and test-retest reliability.
3 Validity: Is there a good fit between an operationalized definition and the concept it is purported to measure?
4 Sensitivity to change: Because the COSP Multisite Research Initiative will be investigating change in 4 month intervals, is the instrument reasonably sensitive to change during that period?
5 Reactivity: To what extent does use of the instrument create change within the subject which would be likely to influence results?
6 Cultural competence: Are the tools free of offensive language, and reasonably well understood by potential participants of all cultural and ethnic backgrounds?
7 Time to complete: Are instruments economical in their use of the time of consumer participants and interviewers, maximizing the information return for every minute added to the CP?
8 Burden to consumer: Related to the previous item, does the CP limit burden on consumers by limiting the time to administer the full CP to a reasonable overall limit?
9 Clarity: Does the instrument avoid questions in which responses would be difficult or impossible to clearly interpret? For example, does it avoid
double-barreled questions which ask about two things at once?
10 Consumer involvement in development: Did the instruments involve consumer in the development of the instrument? Instruments which included consumers in the development process may be somewhat more sensitive to issues which consumers feel are relevant.
Of course, it is impossible to maximize each of these criteria simultaneously, and the instrument selection process included many opportunities to weigh these criteria. The process of developing the CP involved extensive involvement of a variety of individuals from each study site, the coordinating center, and the federal representatives to the COSP initiative. Initial steps to outline the responsibilities and timeline were laid out by the Research Subcommittee, one of the three standing Subcommittees of the COSP. To facilitate the process, the Research Subcommittee divided its work among several Subcommittees, including: Demographics/Employment/ Substance Abuse, Satisfaction/Quality of Life, Empowerment and Housing. In addition, ad hoc committees were formed to carry out particular tasks.
Each Subcommittee was responsible for: (1) reviewing the instruments the eight study sites had proposed in their proposals as well as searching out other appropriate
instruments to consider; (2) collecting information about these instruments; (3) weighing each of these instruments on the basis of the 10 criteria listed; and (4) providing recommendations in the form of proposed items for inclusion in the CP to the Research Committee. In some cases, where the Subcommittee was unable to come up with a single recommendation, it presented the Research Subcommittee with several alternatives.
After reviewing pieces submitted by the Research Committee, the Coordinating Center put together an initial draft of the CP for review and discussion by the Steering Committee as a whole. Steering Committee review of the Baseline CP was accomplished through discussion during a number of Steering Committee Meetings, but also through a formal process of generating input on selection of particular instruments. Three polls were taken to review the CP. These were developed by the Coordinating Center, and results of these polls were distributed by the Coordinating Center: The CP Evaluation provided an opportunity for steering committee members to provide feedback on the overall suitability of items and measures for the CPs; the CP Consumer Information Evaluation provided an opportunity for consumer participants in the COSP initiative to provide feedback according to the extent that particular instruments were consistent with values delineated within a consumer research perspective, including relevancy, sensitivity, and involvement of consumers in the development of the instrument; and the CP Scientific Evaluation in which COSP participants with training, education and experience in scientific evaluation of instruments provided feedback on appropriateness of candidate instruments on the basis of scientific issues such as reliability, validity, appropriateness for study group, etc. Final decisions about about inclusion were made through a unanimous vote of the full Steering Committee.
The following domains represented in the CP as described in Chapter 4.1. Though the GFA required the inclusion of only 5 domains, members of the Steering Committee went go beyond that requirement, taking a more comprehensive approach to the added benefit of consumer-operated services. As the populations that each COSP serve vary considerably from rural Tennessee to urban Pennsylvania and Florida, the most critical issues for these populations also vary considerably. Therefore, the final decision of the domains to explore was based on a need to address the full range of issues affecting consumers in the all sites of the project.
The order of the CP was discussed at great length. The steering committee wanted to create a user-friendly instrument. The document begins with demographic items that open up conversation and that do not present too many difficulties. Items of greatest interest to consumers are then addressed (i.e. employment, finances and entitlement, housing and satisfaction). The next section deals with some of the more difficult items to discuss (i.e. social inclusion, social acceptance, discrimination, quality of life and symptoms). The steering committee discussed the importance of ending on a positive note and therefore the instruments that address recovery, meaning of life and hope conclude the CP.
5400
Arsenal Street