|
Winter 2005 Edition
January 24, 2005
As would be expected in a rapidly evolving field, there are few
well-developed, practical and reliable measures being used in
eHealth evaluations. Most of us are creating our own measures,
with little knowledge of what others facing similar challenges
are using. This presents an obstacle to more rapid advancement
of eHealth. If a set of measures that inform us about important
processes and outcomes associated with eHealth could be identified,
knowledge could be more cumulative. This article discusses some
domains that may be of particular importance for eHealth, and
where possible, suggests types of measures to consider.
To advance eHealth, and to satisfy critics and policy makers,
we are going to have to address some tough questions. These ultimately
boil down to issues of value - are various eHealth innovations
worth the considerable investment in time and resources, and how
large of an impact do they have on issues important to decision
makers and public health? I suggest four broad areas of investigation,
and types of measures that can be collected to address each issue.
1. Who participates in eHealth programs?
We have all heard the criticism that “eHealth will only
work for the small segment of the population that is affluent,
highly educated, and computer savvy.” The various digital
divides in computer and Internet use are closing, but the following
is an important question to ask of eHealth applications: “what
percent and types of persons (and organizations) will participate
in a given program?” This question has direct implications
for whether eHealth will help to close, or will exacerbate, existing
health disparities. Particular characteristics along which it
is important to evaluate representativeness of eHealth participants
include education, income, age, gender, race/ethnicity, computer
experience, and health literacy, numeracy and status. It is also
important to ask parallel questions at the setting or organizational
level - which types of health care systems, practices, and staff
will use eHealth applications? What are the most common reasons
for not adopting eHealth opportunities among those that decline?
2. What are the outcomes of eHealth applications and how
do they come about?
This question seems intuitively obvious, but often is asked in
only a very narrow way. Besides direct effects on a specific dependent
variable, we need to assess issues such as patient (and clinician)
satisfaction, empowerment, health status and quality of life.
One of the lessons learned from medical-surgical interventions
over the past decade is that there are often unintended negative
(and sometimes positive) consequences. Is this true of eHealth
also? eHealth projects should collect measures of outcomes such
as health related quality of life, patient-clinician communication,
or other quality of care (e.g., HEDIS) measures in addition to
the directly targeted primary outcomes.
3. What does the program cost to adopt or implement?
Most eHealth programs are expensive to develop, although if scalable
the incremental cost may be relatively low. But the key economic
question to be answered is, “What would it cost another
setting (e.g., health care system, medical office or potential
purchaser) to adopt or replicate the program?” Costing principles
should follow standard procedures recommended by health economists
and include costs related to infrastructure, equipment, IT staff
time including testing and maintenance, recruitment and training
of staff and patients, and program implementation. The cost perspective
adopted (e.g., purchaser, medical practice, societal) will vary
depending on the question to be answered. Such relatively straightforward
cost estimation is not that burdensome (compared to more comprehensive
economic analyses) and should be routinely reported in eHealth
evaluations, since it is probably the most frequently asked question
about successful interventions.
4. To what types of settings, patients, and issues does
this eHealth program apply (and by implication, for what settings,
patient groups and problems is this program NOT likely to apply)?
This generalization question is one of the most important and,
unfortunately, least often answered questions about eHealth programs.
The good news about such questions is that measures to address
them do not usually require much additional patient or implementation
staff burden. If evaluators keep track of issues such as the types
of settings and patients recruited (and excluded), the characteristics
of those who participate and those who decline, and include analyses
of potential differential impact attributable to setting, patient
and problem subgroups, these applicability questions can be addressed
(see http://www.re-aim.org).
More data on the four issues above would be of enormous benefit
to potential adopters, organizational decision makers, policy
makers, researchers, and citizens. Broadening our focus to include
these issues will advance the state of evidence on eHealth, and
also accelerate appropriate adoption of eHealth technologies.
Readers may wonder if it is realistic to address these issues
in addition to the primary questions of interest in a given investigation.
The answer is yes, some measures or estimates of impact on each
of the above factors are feasible to collect in most eHealth evaluations.
There are both increasing numbers of reports containing such information,
and evolving guidelines (http://www.thecommunityguide.org/;
http://www.re-aim.org)
for such measures.
Russell E. Glasgow, Ph.D.
Senior Scientist
Clinical Research Unit
Kaiser Permanente Colorado
Dr. Glasgow and colleagues have partnered with the Health
e-Technologies Initiative and its Web Portal Grantees to develop
a set of common patient-related measures.
Click here to learn more.
The views expressed in this article are
those of the author and do not imply endorsement by The Robert
Wood Johnson Foundation or the Health e-Technologies Initiative.
|
