Site Updates
Tennessee: Bridges
Tom told the group
that they continually review the values that were presented at the first
meeting eighteen months ago. Every
decision made is based on maximizing consumer involvement for the growth
and power of scientific integrity and validity.
Consumer interviewers, staff and leaders are hired, and are include
them in the decision-making process.
There is a special focus and concentration on rural
populations in Eastern and Middle Tennessee.
A Bridges book is in press from Tennessee’s DMH.
There are many different communications techniques being used to
keep this geographically challenging project fully informed.
There is periodic E-Project News.
CAP membership is geographically dispersed, with 12-14 members.
Scheduling is difficult and some of the meetings are by conference call.
They also use a lot of electronic communications.
The CAP is very involved, hard working and knowledgeable.
Tom is looking for a way to pay honoraria to them.
Irene gave some background on the Bridges program, both in
Tennessee and nationally. The
enhancement money is being used for capacity building, including training
and transportation. Sita
explained that they piloted the common protocol in August and the
induction process and the electronic pilot in the fall.
They have had an 86% retention rate.
Their goal is 360 people for cohort 1.
Some of the lessons they have learned are:
·
TMHS staff is too busy to be much help in recruitment.
·
Transportation is a major barrier.
·
We can actually do this!
Tennessee works with agencies to create lists
of eligible clients for recruitment.
They have a functional assessment that is done on Tenn-care
enrollees for cost data purposes. Lists
are randomly ordered, with the client by phone.
Clients are informed about the study and their rights before
sending the information on to the interviewers.
TN has completed 146 baseline interviews.
Sixty are pending. Interviews
take between two and six hours. Data
procedures include use of a mini-repository in Michigan, a daily ZIP
backup, with data sent to MI weekly.
Michigan backups to CD-ROM weekly, both on and off site. They also
backup to a secured network drive. The
data is then transmitted to MIMH.
The cost study for Tennessee is elaborate.
For the COS - Bridges program TN will be looking at attendance
records as well as detailed program budget sheets.
Irene accounts for all donations.
TMHS cost data is taken out of encounter records, Tennessee care
records, and a managed care twist (no cost, because it is not a FFS
agency, so cost is hard to ascertain). Some of the challenges TN faces are communication,
transportation and delay from baseline to COSP entry.
Sally asked about the
COSP composition. Louetta
responded that there were representatives from traditional agencies,
consumer advocates, and a wide range of individuals from various
backgrounds. Crystal asked if there was a conflict between the two-hatters
– the people who are project staff but are also listed as TMHS staff and
are not compensated for their services.
She was told that the two-hatters do not provide services
Tennessee has found that there is rarely a problem
with interviewers. They feel
the evaluation of the interviews they use has been helpful.
Some interview subjects were believed to be suicidal; therefore, an
incident report was completed, and the providers were notified.
Illinois
The GROW Project has three traditional sites in
Chicago (Tinley Park) in the Center, with Rockford and Champaign (Provena)
on either side. Janet Wattle
Center in Rockford has a long support history.
Personal interviews were conducted with GROWERS to
make sure that the research and various other things do not influence the
program. The qualitative study has been beneficial for researchers in
discovering things about the components of GROW.
It has also helped GROWERS feel empowered and needed.
The IRB has been approved for the cross-site study.
Start-up dates are on target.
The outcome study showed that Illinois is doing the Common Protocol
in the same way as COSP. They will eventually go with the computerized version.
GROW has a long track record with an established program.
That makes for a research friendly and easier way of measuring
fidelity.
Illinois uses four site-specific measure
studies. The GROW Recovery
Scale with sixty-seven items, the BPRS scale, the Social Network
Interview, and the GROW Knowledge Scale (measures the educational program
components).
Their IRB questioned if the interviewers had
the capacity to give Informed Consent.
Training of interviewers is specific for this.
The subjects are questioned to be certain that the consent is
informed. To ensure fidelity,
Interviewers have to complete two taped interviews.
They are paying practice interview subjects.
One of their interviewers has met quality standards already and the
rest will by next week. They
don’t have the confidentiality document yet.
Traditional providers are not recruiting.
They do have fliers and have done mailings.
They offer certificates for shopping rather than cash.
Research participants get two orientations: a research orientation
and a GROW orientation. If
they are still interested after these orientations, Informed Consent and a
release of records is obtained. The
baseline interview takes 4-7 hours because the site-specific interview is
about as long as the COSP Common Protocol
The cost studies are being conducted in two sites –
Champaign and Rockford. The
Cost Study is probably going to be a Track 2.
Tennessee is working on getting costs for donated spaces.
Entering and Sending Cost Data:Next Steps
Matt Johnsen
Matt stated the news of the Common Protocol, as
a whole is good. One hundred
and sixty-two interviews have been completed and that represents about 6%
of baseline. There seems to be some confusion about what constitutes a
Track 1 or a Track 2. He
noted that some sites reported being able to collect Track 2, but
expressed reluctance to commit to this track.
He
reported that PIs are interested in getting the cost study going. There are two ways to approach this: full speed ahead, or
step back and reflect on whether this is a common vision, with common
goals.
Among the questions asked were: To what extent will
data from cost study be pulled together with different sites using
different procedures? While
using common metric across all sites ($) if there are different tracks
with different sources of records, how reliable is the data?
If the same method is being used on both sides, a comparison
between TMHS and COS can be made. However,
reliability of pooled data is not as clear-cut.
Before we get in the field, R.O.W. has asked for an outside expert
(MH Economist) to review current approaches, to insure that we are using
scientific rigor. Matt says
this is not an uncommon situation in cross-site studies (cost studies
across sites). Solving this problem could allow us to take a leadership role
in this and future studies. The
cost study is required, so it will not go away.
We need to recognize the efforts of cost study leaders, and
recognize that this is the first time this has been done in MHS, and very
few other studies. The cost
study is and continues to be GFA required.
Matt said we need to have buy-in from all of the sites and we need
a consensus to get started on this. This requires a full commitment from
all sites. He indicated that PIs need to get together to form an
approach with full cooperation of all involved.
We may need to set bar lower for all sites, allowing individual
sites to collect what they want, but we must have common data from all
sites. Twofold
responsibilities here: Honesty in approaching issue with less
defensiveness, and honesty in raising and clearly articulating concerns
that exist. If we are truly
collaborators in this effort, it is imperative we refrain from sitting on
our hands. Matt expressed the
need to focus more on scientific process and less on personalities.
He said there was a need to keep our eyes on the prize, and the
cost effectiveness of COS. Ultimately,
this will have a great impact on consumer services and what will happen in
the future. The steps we take
today will preserve the integrity of the program.
Ruth said everyone mentions that they are
collecting costs. The issue
is what are we collecting. She questioned why she wasn’t hearing any
common ingredients to cost studies. She
feels that in order to have a cross-site, cost-study the PIs need to have
a CCP – a Cost Common Protocol. She
thought there might even be two studies, a limited common cost study and
an analysis cross-study.
Randy said that adding cost studies to research
adds a political dimension that should be avoided.
Results become one of compliance, not individual participation.
Zahira feels that CAP was excluded from the
cost study decision. She
stressed that the research is about the COSP and it is meant to promote
the COSP. She noted that
whenever decisions were going to be made about the cost study, CAP and the
consumer representatives should have been involved and polled on these
issues from the beginning.
Matt responded by saying that he has always
tried to adhere to the same procedures of site inclusion and dissemination
of information. He thought
gleaning information would be helpful and arranged a luncheon with other
PIs to discuss their experiences. Several
issues of concern arose from the meeting.
Matt asked the CC for time to address these issues.
Diane explained to Zahira that no decisions have been
made as yet, just discussion. If
we do this, the burden on sites has to be equal and some technical
assistance needs to be available in this area if needed.
Jean encouraged communication about all issues
in order to work on levels of trust and knowledge.
She said we are trying to measure the cost for the experimental and
the control groups as they enter our study.
As we induct people and begin our fieldwork, we should concurrently
be doing the cost study, as it will be easier to capture all services at
this time. Everyone needs to
do his or her research. The
cost manual is on the Freedrive. Decisions
made have to be done procedurally, in order words, if we change course we
need to have a two-thirds vote of this group to reconsider.
We also need to vote on what our cost study will be.
This means it involves the Steering Committee, as agreed.
Tom reiterated that we are cutting edge – no one
else has gone before us. We
need to ease up on ourselves as we learn, and be careful not to be overly
critical. It would be helpful
for us all to make a matrix that describes what each site plans to do.
Yvette said it is too busy at her agency to take on
additional responsibilities of factoring in costs. She expressed concern that her program will look bad if they
have to add costs to volunteer and in-kind donations.
Tom asked the group to recognize the efforts of Brian
Yates, Cost Subcommittee.
The times for the June Face-to Face meetings are from
12 Noon on Sunday to 3:00 p.m. Tuesday.
Subcommittee Reports
Logistics
The Logistics
Committee met and made the following recommendations:
That a
quantitative rating of the responses by the consumer be done at the end of
each interview and follow-up. Emphasis was placed on not rating or
judging the person but judging the quality of the information received. Margin comments might be useful for training and interviewing
but are not a substitute for doing the quality rating. The Interviewer should check sections that are
Every attempt should be made to include people
who have a guardian unless it is going to cause problems with the site’s
IRB process. Motion was made and passed.
problematic. A revised Interviewer Rating form was
presented. It is a
three-point scale showing how to assess the reliability of the data based
on consumer responses.
Motion was
made to adopt the Interviewer Observation Rating Scale. Motion passes.
Each site must have pledges of confidentiality.
Motion is that all sites are required to have
pledges of confidentiality by the interviewers and research staff.
Motion passes.
Points of information:
·
Illinois passed out copies of a structured assessment they
use for getting informed consent.
·
Two sites have to re-review informed consents at each
assessment point annually due to their IRB requirements and one site,
Florida, is just an oral re-review. Most sites don’t have to do a
re-review and this will be left as site-specific.
·
Study-Blind was shelved until a later date.
The
recommendation is that sites must track the date a person first attended
COS, last attended COS, or withdrew from the study, not the intervention,
whether a site uses the Tennessee database or not.
Also recommended is those sites track data for the reason people
stopped going, didn’t go, or that the assessment was not completed.
Common Ingredients
A book outlining the Common Ingredients of Consumer
Services was developed. Contact
has been made with two publishers. They
are looking for a new title. The
Committee discussed the instrument of recovery and the need to clarify
language; however, they have no recommendations to make at this time.
CAP
The CAP Committee met and discussed the
Communications policy. They
recommended that the Steering Committee exempt the CAP from the open COSP
list. They requested that the CAP list consist of CAP members, the CC, and
two Federal representatives.
Some
objections were voiced to the recommendation for closing the list.
The reason given was that CAP members need a place where than can
discuss issues in private, come to consensus, and form reactions to other
things that are going on in the list serve.
Anyone who wishes to post to the CAP list could
e-mail Zahira or Sally and request that it be posted by them.
Jean said that if the list is closed, the CC should receive a
monthly summary of the main discussions or major decisions.
She added that if something critical went on the Steering Committee
needs to be advised. Zahira
said the CAP could accommodate this.
A motion was
made to close the CAP list and close the archives to limit the readership
to the CAP, the CC, and the two Federal representatives.
Motion passed.
Publications/Presentations
Pat thanked Jean and Rita for publishing the first
newsletter. He asked if the
Committee might give them a sense of the mission of the newsletter and an
idea of the regularity of it.
The following revised individual site
disclaimer was read:
“The contents of this paper are solely the
responsibility of the author and do not necessarily represent the
officials of DHHS, SAMHSA, CMHS or the other collaborating partners.”
A motion was made to accept the disclaimer as written.
Motion passed.
A
recommendation was made that no papers should be submitted on the Common
Protocol as outcomes, or on Common Protocol costs as outcomes until the
foundation papers are accepted for publication. What this means
according to Pat is that the Common Protocol data is open for analysis.
Pat reinforced that all other papers are welcome.
Pat explained that since there was a timeline written in the policy
on foundation papers there was no need to include it in the
recommendation.
Motion passed.
Tom
said that there were two issues concerning the follow-up Common Protocol
that people wanted to reintroduce. There was a motion that the COSP satisfaction and outcome
section come before the mental health question and the follow-up Common
Protocol. The original vote
was to have the COSP questions asked first and then the traditional
questions asked second. Tom
thought this issue should be reopened and possibly the group should go in
another direction.
Greg explained
that only people in the TMHS would not have had the COSP experience.
Those who get the COSP first will have had a different experience.
There is a potential for a contrast in cost for differences in
satisfaction for Consumer and traditional services.
He recommended
starting everyone out on a level playing field.
Motion was
made to reopen this issue to vote on.
Motion passed by two-thirds majority.
Members voted
on the issue of whether to switch back to having the traditional
satisfaction of outcome questions of MHS precede the COSP outcome
questions. This vote passed
by two-thirds majority.
Next Steps
Tom will get a small group together to make up
a project-wide timeline. The
PIs gave Tom the date they started or intend to start their site’s
baseline data collection. Tom
will be putting up a user’s manual for the tracking database that can be
downloaded from the Freedrive. Sally
will send out a list serve discussing additional tracking fields,
including addendums. Jean
mentioned that the officially approved timeline was in the packet given to
each F2F participant. She
asked that the group review the timeline because it will need to be
updated.
Sally asked where interviewers could get help or
information if they need it. Jean
said Kim Einspahr is the coordinator of the Multi-site Interviewing
Protocol. She maintains a
list serve and will answer questions after consulting with Jean.
Jean said some decisions would have to be referred to the Steering
Committee. Tom said a log of
the decisions made, as well as a manual for disseminating that information
would be helpful.
Tom asked for feedback on the electronic voting
process. The group agreed
that the electronic voting was simple and easy to use.
Brian stated that it had made a big difference in the quality of
this F2F experience.
Meeting Wrap-up
Jean asked that the evaluation forms be filled out.
She noted that the June meeting would start at 12 noon on Sunday
and end at 3:00 p.m. on Tuesday. She reminded the group the voting would continue until the
end of the meetings; therefore, people should not plan on leaving early on
Tuesday.
The meeting was adjourned.
5400
Arsenal Street