Steering Committee Meeting Notes
September 10, 2000
12:00 p.m. – 6:00 p.m.
Embassy Suites
Washington, D.C.
Welcome
Pat Corrigan, the Consumer-Operated Service Program (COSP)
Steering Committee Chair, welcomed the new faces, and had those who were
attending their first meeting introduce themselves. He then read the following five basic principles shared with
the SC by the Federal Representatives:
Focus on the
issue, not the person.
Maintain the
self-esteem of others.
Maintain
constructive relationships.
Talk the
initiative to do things better
Lead by example.
Pat presented a humorous program on what makes a meeting
effective and successful.
Grants Management Update – Betsy McDonel Herr, GPO
Crystal Blyler, filling in for Betsy McDonel Herr,
described the changing strategy of grants management. The SC was asked to direct
their calls and complaints directly to Steve Hudack of Grants Management, or for
further discussion to his boss, Christine Chin. Refer to your NOGA (Notice of Grant Award) for her name and
telephone number. Remember to carbon copy your comments and questions to Betsy
McDonel Herr or Crystal Blyler so that they are aware of your talks with Grants
Management. Grants Management
doesn’t fall within DMHS, they fall under SAMHSA.
Pat Corrigan asked if the PIs would want to write a letter
to Steve Hudack regarding the frustration the COSP project is having.
Susan Essock said she would bring it up at the PI dinner.
Sita Diehl stated that it was difficult for vulnerable organizations that
serve vulnerable people to engage Grants Management in the way described by
Crystal. Tom Summerfelt said that
he has frequently tried to communicate with Steve and this has almost always
resulted in no response. Crystal
said he almost never responds to e-mails or messages.
She added that you have to catch him when he is in, and suggested that it
is better to call early in the morning.
Tracking Enrollment and Other Performance Indicators – Rita Adkins, CC
Project Manager
Rita Adkins explained that
there is now one data entry program that accepts both Baseline and Follow-up
data. The data has been converted,
and all sites are compatible with this latest version. She reviewed the improved procedures for implementing
changes in the CP and FUP. All of
the database changes and examples are included in the F2F packets in the “Data
Repository” folder. If you have
questions, call Rita Adkins or Keith Eldridge at the Coordinating Center.
Some revisions have been made to the Quarterly reports at
the suggestion of the Federal representatives, the sites, and input from the
Logistics Committee. Fields from
the Tennessee Tracking database were used. Definitions of fields are needed to
ensure that everyone is reporting on the same criteria.
The definitions are:
·
Recruitment – Those people who are referred to the study and
have agreed to be contacted
·
Enrollment – Participants have consented to be part of the
study, have completed baseline, and have been randomized to either the
experimental or control group
·
Withdrawal from study – The participant informs the research
team that they no longer wish to participate in the study.
Sally Rogers thought it would
be better to say “withdrawal from research” rather than “withdrawal from
study.” The issue on definition of withdrawal was referred to the Logistics
Subcommittee.
Rita Adkins defined recruitment
attrition as the loss of respondents to enrollment. It will be calculated from the difference between those who
give consent to be contacted and those who complete enrollment. Enrollment
attrition is defined as the loss of the respondent to the research.
It will be calculated from three different areas: 1) those who enrolled
but withdrew from the research, 2) those who are randomized to the experimental
group but never engage the COSP, 3) those who are lost to follow-up.
Definitions are still needed that will require a decision by the SC based
on the Logistics Committee recommendation.
They are attending the COSP, withdrawal from COSP, loss to follow-up, and
crossover to experimental.
Rita went through the complete
Tennessee tracking database fields, and identified the areas where the field
information for the Quantitative Quarterly Report could be accessed.
The various components of the report are: 1) the work plan which is
required, and is based on the timeline for the original submittal, 2) the status
of the study participants, 3) core cost study checklist, 4) enhanced cost study
checklist, 5) the COSP Steering Committee activities, 6) the plans for next
quarter.
A number of key indicators need to be monitored.
The Flash Report is a mini status report that measures recruitment,
enrollment and attrition on a weekly basis. The Federal representatives
requested this report. The report allows for pro-active responses to any
problems that are identified. The Flash Report is due at the CC on Thursdays at
5:00 p.m., PT. Submissions will not
be accepted after that time. The
report will be aggregated by site from the running total, and forwarded to the
Federal team on Friday.
Diagnosis will be monitored. An Axis 1 diagnosis is a
severe mental illness of sufficient duration to meet diagnostic criteria as
specified in the DSM 4, and/or that has resulted in a functional impairment that
substantially limits or interferes with one or more major life activities.
Additionally, an Axis 1diagnosis is substance abuse disorders, developmental
disorders, and “V” codes. The
SC determined that an Axis 1 diagnosis would meet the GFA criteria. Sally Rogers
objected, saying that the SC had not come to that agreement. Jean Campbell
stated that the requirement was that we collect and analyze Axis 1 diagnoses to
make sure the GFA criteria is being met; however, we will be collecting and
analyzing all four Axes.
The Logistics manual states that after three months of data
collection, analysis should be conducted to see if the recruitment, as specified
by the GFA, is working. As of June
30 there were 341 cases submitted and aggregated.
Only one of those had a diagnosis, and a decision was made that the
diagnosis must be submitted before the FUP data is transmitted. If the FUP
doesn’t have a diagnosis the repository will reject the FUP as incomplete and
the enrollment figures will be ejected to reflect this.
That gives a three to five month window to collect this information.
The repository will be monitoring enrollment.
The enrollment figures come from the site’s cooperative agreement and
from the number of study participants that were included in their original
proposal. The CC is unable to
change these numbers because it is under the authority of the Federal
Representatives. The CC will be sending an aggregate of the enrollment and the
percentage completed for each site. The data will be collected on both the
control and the experimental group. Reports
will be sent to the Federal Government.
Rita Adkins announced that 67% of the projected baseline
has been collected as of the 3rd Quarter. The summary of items to be
tracked and reported to the Federal team by the CC are the BL and FUP
enrollment, waves 1, 2, & 3, recruitment, attrition, and diagnoses. Time was
calculated based on the different waves. Jean
Campbell reiterated that the enrollment times are based on last available
quarters for yearly reports.
Rita thanked Sally Rogers and
Ruth Ralph for their help.
There were many objections from sites about the weekly
Flash Report. Complaints centered
on the unrealistic demands on time and money. Diane Cote, Yvette Sangster, and
Susan Essock thought discussions and mediations on these unrealistic demands of
time and money should have taken place before the demands for implementation of
the weekly report took place.
Betsy McDonel Herr and Crystal
Blyler explained that the data is critical for tracking what the sites are
doing, identifying problems, and providing early intervention.
Betsy said that this project is not as far along as it should be, and
stated that the Federal team can’t stand by and wait to find out if there are
severe problems. They need to look
for trends in order to give warning notices.
Susan Essock added that what
was coming across in the Flash Report presentation seemed like a threat to her,
and challenged the language used on the Flash Report that sites not turning in
their reports on Thursday by 5:00 p.m. would be reported to the Federal team.
Tom Summerfelt mentioned that he felt like the SC was ambushed and that
it would have been better to know before this meeting that the Flash Report was
coming down the pike rather than digest this sudden surprise.
He asked that in the future, open discussions take place via the
listserv.
Betsy McDonel Herr and Jean
Campbell stressed that the Flash Report was not intended to be threatening, but
should be consider a management tool to see where the study is. What the Federal
team needs is more collaboration, and to see the numbers going up.
She said they could get technical assistance when problems are
identified. Jean emphasized
that timing is an important factor since the CC will have a tremendous amount of
work to do before sending the report to the Federal Representatives.
Querying Site Data – Tom Summerfelt
Tennessee had 278 baseline
interviews completed and verified. Tom explained the facts and figures that he
had gathered on baseline comparison to see if randomization worked at their
site. He has posted it on the freedrive if SC members would like to see it.
Tom made the following recommendations:
·
cleaning is the name of the game
·
begin working with the data ASAP
·
look at frequencies, distributions and missing values
·
plan frequent and routine data checks
Betsy acknowledged Tom Summerfelt’s major contribution to
the multi-site. Rita Adkins
mentioned that Keith Eldridge at the CC has information on how to collect and
summarize data.
Presentation of Site Tracking Data
Tennessee - Tom Summerfelt
Tom Summerfelt said that TN contacted over 2000 people.
He reported that 497 people gave consent to be contacted, 367 gave
consent, 104 refused to consent, 18 were excluded from the study due to
ineligibility, 310 started baseline interview, and 3 aborted interviews. In
response to questions about the aborted interviews, Jean Campbell explained that
no one is enrolled in the study until randomization has been completed.
Those people would be considered the recruitment attrition.
Tom reported that there are 151 in the control group and
156 in the experimental group. The
total is 307 for the completed baseline. There
are 68 people in the randomized to experimental and attending the COSP group.
Sixty-six people are in areas where the classes have not started.
Therefore, the truth will be between 68 and 124.
Ninety-three individuals started assessment at first follow-up.
Seven people have withdrawn from the study.
Connecticut - Linda
Dunakin
CT has eighty-three consents of which eighty-one were
actually eligible. Linda Dunakin
reported 18 refusals to contacts, 53 gave consent, 0 refusals, 28 individuals
were randomized to the control group, and thirty-two were randomized to the
experimental group. Only two people
in the experimental group had never gone to the COSP.
Nine people did not finish the program.
FUP assessment at four months showed fifteen in the control
group and fifteen in the experimental group.
Pennsylvania - Dena Plemmons
Dena Plemmons reported the following data for study
participants: 22 people consented, 19 agreed to participate in the study, 3
refused, and 17 completed a baseline interview. Two people signed the consent, but chose not to complete it.
They have four completed assessments at the FUP.
Seventy-seven percent of the individuals eligible for the study are in
the study. Referrals to COSP have been low.
Case manager turnover has been significant, and new case managers need to
be educated. Presentations are made
to case managers every other week. Study
brochures and the Office of Mental Health has sent out information to the
agencies encouraging referrals.
FLiCA – Greg
Warren and Nancy Erwin
Nancy Erwin reported that Florida started out with three
hundred ids in the database, and California had 144 ids.
Only 22 baselines have been completed in the two locations. Two problems
identified at the TMH site are that the top down administrator and line staff
don’t always agree, and the staff at one site is unsure about what to do with
dropouts, and no appointments. More
than 60% of their numbers were meds only people.
There is no way to contact these people, and a lot of the people don’t
fit into the database.
Matt Johnsen asked why people were ineligible for the
study. Nancy Erwin said they had
been to the COSP too many times. Diane Cote said FL is a transient state with a
lot of homelessness.
Maine – Ruth Ralph
Ruth Ralph reported the site tracking data as follows: 115
consent to contact, 1 ineligible, none of the 115 has refused to meet.
Sixty-four people have signed the informed consent.
Maine has been through the SPA process, which means that they had to stop
interviewing for 2 months. Sixty-four
baselines have been completed with nine assigned to the control and twenty-five
to the experimental group. Three of
the experimental group are attending the COSP.
Attendance is done through the COSP database.
MCAP advised MA to set up a buddy system so people can go
to the COSP together. COSP sends
welcome letter and invitation. Kathryn
Kidder makes presentations at the case managers meetings. Posters are put up in case management waiting rooms, and in
agencies that refer people to case management in order to get them into the
study.
Jean asked the speakers to give the CC a copy of the
information they presented.
Missouri – Sally Rogers and Brian McCorkle
Sally Rogers reported that 138 individuals have been
invited into the study, 83 individuals were contacted, 45 have agreed to be in
the study, 38 have refused, 21 have signed an informed consent, and 8 have
refused to sign. Twenty-one
baselines have been completed. Twenty-one
individuals have been randomized – nine to the experimental group with only
one person attending the COSP. There
have been no withdrawals and no crossovers at this time.
Brian McCorkle said 16% of the people refused to be contacted, 32% have
said yes and interviews are being scheduled. They are waiting to hear from 36%
of the people. Sixteen percent of
the baselines have been completed. One emerging pattern is the clients who
decline to be in the study. African-Americans
and women are declining at a higher rate than men are. There was a small
discussion about a “soft” yes being responsible for the change of heart when
the client is actually contacted and refuses to join the study.
Illinois -Sarah Lickey-Diwan
Sarah Lickey-Diwan reported 109
statement of interest forms have been filled out, 99 people are eligible for
inclusion, 5 are not eligible, 6 are unreachable at the phone # given, 0
refusals to contact, 70 people filled out an informed consent, 1 person refused,
1 person dropped out of study, and they have not been able to reach 29 people.
Fifty-seven people have completed the baseline, 27 in Champaign and 29 in
Rockford. Of those, 27 are in the
control group, and 30 are in the experimental group.
Twenty-six people were randomized to experimental but not attending COSP;
four people were randomized and are attending. Two FUPs have been completed.
Sarah said that their new technology is to pass candy out
at an information booth and tell people about the study. They are screened on the spot, which cuts out a step.
Illinois does not use case manager referrals. Interviewers hang out in
the TMHS area.
Date-Driven Discussion: Problem Solving Collection Issues
Pat Corrigan, SC Chair, and Rita Adkins, CC Project Manager
Pat listed the following
issues, and asked the SC if there was any that could be resolved immediately:
·
Randomized to COSP (not going) ideas: Pat said this would probably
be a site by site issue.
·
Concerns about recruiting people of certain demographics
·
The issue of definitions (Decision by SC - Send to the Logistics Subcommittee)
·
Flash Report (Jean
Campbell spoke with Betsy McDonel Herr, and a compromise was agreed upon for the
Flash Report timeline.)
All sites will complete a monthly Flash report. Weekly site reports will be completed to collect data on how
many people were randomized. Tom
Summerfelt removed his objection to enrollment data being done on a weekly
basis. Consensus reached.
Susan Essock and Sally
Rogers objected to the wording at the bottom of the Flash Report that stated
sites would be reported to the Feds if their report wasn’t in by five
o’clock p.m. on Thursdays. Susan
E. stated that she strongly encouraged that communications to the SC, rather
than being worded in a punitive language, be worded in a less threatening
manner. She suggested the wording
be: “Submissions after that time will
be included in the next week/month’s report. If the timelines statistics
indicate that one or more sites is having difficulty meeting this deadline the
CC and/or the Feds will work with the sites to see how these difficulties might
be overcome.”
Betsy McDonel Herr moved that Susan’s suggestion for wording is adopted as
stated.
No objections, motion passed. Susan
said we should try for this kind of language on an ongoing basis.
Matt Johnsen reminded the SC that larger numbers of people
are needed to participate in the COSP in order to know what added benefits
people in the study are getting from participating in the COSP.
Betsy McDonel Herr said we might get more numbers but risk not getting
the benefits. She and Mike English,
agreed that the multi-site needs to be extended which means going through
another GFA. Betsy and Crystal Blyler are trying to get additional monies.
A resolution is needed by Jan 2001.
The Baseline ends March 2001.
Brian McCorkle wanted to know why MO is doing randomization
selection from providers, while PA and CT are doing self-selection.
Matt Johnsen said those sites started with a different target population
and a different selection process. A
discussion ensued about people being lost because randomization does not work
with COS. Pat Corrigan wanted to
know what do we do with non-randomized design?
Jean Campbell suggested writing a recommendation with documentation about
what needs to be done, and send it to the GPO.
Betsy McDonel Herr said they may come to same conclusion of the COS, but
it will be three months down the road. Pat
Corrigan made a motion to revisit this question in three months.
Rita Adkins offered a friendly amendment to revisit the question in four-months,
instead of three months.
Betsy reminded the SC that there were no guarantees.
She told the sites that if they can’t do it, there is no mandate to
keep the study going. There will be
a close out of the study. A better
thing would be to write a new study, a new GFA, and a new mandate.
Sarah Diwan moved to
make a concerted effort to analyze reasons for non-attendance.
Helen Minth seconded the motion. Jean
Campbell stated that tracking from the monthly database that goes to the
repository answers questions about which people are attending the COSP, and
which people are not.
Pat Corrigan announced that the motion on the table was that in three months
the CC would provide data on how
many people randomized to COSP are actually going. Data will be analyze, site by site, to find the reasons for
the non-attendance. Tom seconded the motion.
Motion passed.
Motion to table
until January cutting out the diagnosis requirement of one year of mental health
services as required by the GFA. Motion
passed - no objections.
The motion to recruit more clients of certain demographics (e.g. males) was
tabled.
“Definitions” were referred to the Logistics Subcommittee for nuancing.
Quarterly Report
Brian McCorkle wanted to strike
the tracking of intrasite meetings from the report. Crystal Blyler explained
that the Federal team has had complaints that people are not communicating or
getting information from others within their sites. Betsy McDonel Herr said the Federal representatives need to
show links and appropriate communication between the CAP and the researchers.
Tom made a motion to delete the
whole page from end of the quarterly report.
Logistics Subcommittee will study how to gather diagnosis data and make a
recommendation.
Agenda items, “Ways to clean the data” and “Enrollment”, were
tabled.
5400
Arsenal Street