In addition to their participation in training by ROW Sciences in the site visit protocols and procedures, team are also expected to have prepared for each site visit by reading materials provided by each site describing the COSP and traditional program(s), including:

The original proposal for the site being visited.

The site-specific logic model.

The site’s most recent quarterly report.

A list of services offered by both the COSP and the traditional site(s).

Additional materials sent by the site (e.g., descriptive brochures, mission statement, curricula, leaders’ manuals, bylaws, policies and procedures, rights and responsibilities, grievance procedures, etc.).

Agenda. In general, the equivalent of 1 full day is needed for each organization/agency visited. However, the agenda for each site visit varies according to the needs, characteristics, and schedules of the local organizations/agencies. A contact person is designated by each site to work closely with ROW Sciences to identify key personnel to be interviewed, to schedule meetings with targeted interviewees, and to make logistical preparations that address the needs of both the site and the Coordinating Center.

A typical site visit (see Sample Site Visit Agenda, Exhibit 7.1) consists of separate interviews or focus groups with the following individuals and groups:

· For the overall site. (Focus on issues that might affect the research protocol, results, or interpretation of the results.)

— PI and relevant research staff. To discuss the progress of the initiative and anticipated problems.

— Consumer Advisory Panel. To discuss the issues above, as well as the initiative’s responsiveness to consumer input.

— Exit interview with the PI. At the PI’s discretion, an exit interview can be held to report general observations and provide an opportunity for greater clarification.

· For each COSP and traditional service visited. (Focus on how services are implemented, and similarities, differences, and interactions between COSPs and traditional services, and different COSP models. Explore any differences among interviewees’ perceptions of program functioning and actual functioning.)

— Director of organization/agency. To discuss the organization and its services, the local service network, and how the organization interacts with the network, especially the COSP or traditional service(s). This should also include a tour, with the opportunity to observe program activities, as appropriate.

— Supervisory staff interviews. To discuss the organization and its services from the managerial perspective. To gain a better understanding of similarities, differences, and interactions between the COSP and traditional service(s).

— Line staff interviews. To discuss the routine operations of the program from the direct service perspective. To gain a better understanding of similarities, differences, and interactions between the COSP and traditional service(s).

— Consumer focus group. To discuss participants’ experiences with and perceptions of the COSP and the traditional service(s).

— Diverse consumer interviews or focus group. To discuss the aforementioned issues with the goal of determining whether participants experience or perceive the COSP or traditional service(s) differently from other consumers because of their diversity.

Each consumer focus group should consist of 8-10 participants. Group members should be typical of individuals who attend the program. Consumers from the traditional sites should not have attended the COSP. Although diverse consumers are welcome to participate in the larger focus group also, it is important that they are also offered an opportunity to speak to the site visitors separate from the larger group. This should allow people to express their thoughts and experiences without the pressure of feeling different or being judged.

Consumers are paid to participate in a focus group or an individual interview. The amount of money is approximately the amount paid for a common protocol interview at your local site. (Participants sign both informed consent forms and receipts to receive their payments.) R.O.W. Sciences also provides refreshments or a pizza (or equivalent) lunch/dinner for focus groups if appropriate for the time at which the focus group is held.

Exhibit 8-2 contains a description of the consumer focus group process for sites to use in recruiting consumer participants; Exhibit 7-3 depicts a sample informed consent letter for consumer focus groups.

Interview Questions. Each interview or focus group consists of a series of semi-structured interview questions. This format allows the site visitors to skip some questions and pursue other topics in depth.

Confidentiality. Although site visitors record the first and last names of all staff members interviewed, and the first names of all consumers interviewed, responses are never attributed to specific individuals without their permission. Only interviewees and site visitors are allowed to attend interviews or focus groups.