Outcomes research in prostate disease

An informed way of looking at treatment results

The accuracy of prostate cancer statistics, usually the focus of clinical conversations, entered the realm of presidential politics in October 2007. The New York Times reported that a figure cited in a radio advertisement for Rudolph Giuliani, a candidate for president, was in dispute. Giuliani said that his chances of surviving prostate cancer in the United States were 82%. In England, he claimed, his chances of survival were just 44%.

The Office for National Statistics in Britain cried foul, saying that the five-year survival rate from prostate cancer there is actually 74.4%. Doctors weighed in on the controversy, saying that it was unfair to compare survival rates in the two countries because prostate cancer is more likely to be diagnosed in its earliest stages in the United States, where prostate cancer screening has become routine. According to the Times, the disputed figure was drawn from a journal article that was seven years old, and the author of that article acknowledged that his calculations were crude.

All the flap pointed to a much larger issue: Just how accurate are the statistics we hear? How are the numbers calculated? And can patients rely on reported figures to determine the likelihood that they’ll suffer a particular complication — urinary incontinence, for example? Where can patients find accurate information on treatment outcomes? Why do data from one reputable organization vary from another?

To learn more about the issues — and help patients make better-informed choices about treatment — Harvard editors invited three experts from hospitals affiliated with Harvard Medical School to discuss these questions and the growing field of “outcomes research.” The panel included these people:

• Frank L. McCaffrey, LICSW, OSW-C, a senior clinical social worker on the Oncology Service at Beth Israel Deaconess Medical Center. He routinely meets with prostate cancer patients who need assistance in making treatment decisions or counseling following treatment. In addition, he facilitates patient discussions at monthly meetings of the Longwood Medical Area Prostate Cancer Support Group in Boston.
• Rebecca Miksad, M.D., M.P.H., an instructor in medicine at Harvard Medical School, an oncologist at Beth Israel Deaconess Medical Center, and a research fellow in the Program in Cancer Outcomes Research Training at Massachusetts General Hospital. Her research focuses on improving cancer treatment decisions through better characterization of patient outcomes and quality of life, assessment of the economic impact of chemotherapeutics, and the use of decision-analysis tools.
• James A. Talcott, M.D., an associate professor of medicine at Harvard Medical School who directs the Center for Outcomes Research at Massachusetts General Hospital. The center evaluates the impact of cancer and its treatment on patients. He has authored numerous scientific articles about how patients make treatment decisions and the effect those decisions have on their quality of life.

The basics of outcomes research

What is outcomes research?

MIKSAD: Outcomes research is an umbrella term that covers many different facets of health care. In general, outcomes researchers attempt to evaluate the end result of health care — how diseases and treatments affect people’s lives, and what people care about, such as changes in their ability to function, their quality of life, and mortality.

TALCOTT: It’s a relatively new field that grew out of the early observations of researchers at Dartmouth Medical School. They noticed that health care varied in ways that could not be explained by variations in medical conditions and that nonmedical factors contributed to the way we practice medicine. At the same time, medical care was getting very expensive. So, the movement to look beyond the biology of disease to the multiple effects of health on society — and of society on the delivery of health care — began.

How does outcomes research play out in your day-to-day work with patients?

McCAFFREY: I have a great appreciation for outcomes research and what it can tell us, but as a social worker, I’m really down in the ditches with people. Their individual experiences are so variable. For example, two men may have cancer that’s metastasized, but their values and quality of life are very different. There’s definitely value in outcomes research, but I’m not sure how best to explain what it means to the people I see. They’re trying to decide what’s right for them as individuals.

Patients sometimes posit that the incidence of impotence or incontinence is higher than what the literature and their physicians say. Where do the data come from, and why are there discrepancies?

TALCOTT: After PSA screening was introduced in the early 1990s, patients were faced with a wide array of treatments, including active surveillance. And given the location of the prostate, it was evident that depending on the treatment, urinary and fecal incontinence, as well as sexual dysfunction, might result. But there weren’t well-established standards for collecting information. Physicians would informally keep track of their results, asking patients about any problems or making inferences based on things such as whether a patient was wearing pads to absorb urine. Data generated that way seemed to be pretty optimistic compared to what we as outcomes researchers heard from patients.

Did the physicians hand out formal questionnaires?

TALCOTT: No. They made reports based on what patients said during office visits or casual conversation. There are a couple of problems with this method. One is that physicians, like everyone else, are more interested in their successes than their failures; they may not have pushed as hard as an unbiased person to get information about side effects from their patients. On the patient side, it can be embarrassing to talk about such intimate problems, and there’s a desire not to affront the physicians that have been treating them. We found that they would say one thing to the physician, but when we gave them a questionnaire, many more problems emerged. I think that’s the first principle of outcomes research: if you want information about a patient’s condition, ask the patient, not the physician.

MIKSAD: I agree completely. It can be hard to find the time to talk about these issues during an appointment. Something might occur that is of concern, but when the patient comes to an appointment a few weeks later, something else is of more immediate concern. The physician and the patient don’t get the time to address the slightly less-concerning issues.

TALCOTT: Discrepancies also arise when you try to define any side effects or symptoms. For example, how strict are your definitions? A question about incontinence might be, “Do you leak urine or not?” Another definition could be, “Do you leak enough urine to regularly soil your clothing?” People can interpret terms that seem very clear to doctors in significantly different ways.

Once the information is gathered, you need to create and validate a so-called functional scale in which you take the answers to several questions and arithmetically convert them into a scale from 0 to 100. That allows investigators to assign a number to each patient’s condition, compare groups, and build mathematical models that help adjust for differences between patients.

Does that result in more-accurate data than a physician-patient interview?

TALCOTT: Absolutely. The problem is that numerical scale scores don’t mean anything to patients. If you present the data in ways that make sense to investigators, such as the average patient getting treatment X has a quality of life that’s 10 points higher than someone receiving treatment Y, you’re not really telling patients what to expect from treatment. There are really two parts to outcomes research: measuring scientifically and then turning the results into information that patients can use to make decisions.

What do patients complain about when they talk with you?

McCAFFREY: I met with one man who had a prostatectomy several years ago, and he talked about how awful his incontinence was. When I asked him to describe it, he said he might have a few dribbles when he’s active. He doesn’t wear a pad, but he calls it incontinence. Having known plenty of men who suffer from incontinence, I’d say he doesn’t have it. Then I’ve seen other men who go through several pads a day, yet they’re relatively happy because their condition has actually gotten better. So there’s inconsistency across the board, and for something this important, consistency is crucial.

But bottom line, what I hear from patients is that they don’t feel the doctor quite prepared them to run into major problems, such as sexual dysfunction or incontinence. But can any doctor fully prepare someone for what they’ll face? No. And until you, as the patient, are there dealing with the problem, you might not understand the various degrees of erectile dysfunction, for example. Even so, I’m not sure that the doctors are really saying, “This is something you’ll likely have to deal with.”

So patients say that they really weren’t informed that this or that could happen, or that the degree or the frequency of it was somewhat more than advertised?

McCAFFREY: Exactly. Some men might go in and ask for percentages, yet feel that the percentages their physician cited are nowhere near what they read in a book or saw when they looked up information on a reliable Web site.

How do you counsel a patient who comes in with conflicting statistics?

MIKSAD: I always tell patients that I can give them averages based on studies that have been reported. But if you’re the patient who develops the problem, the averages don’t matter anymore. And the patient’s situation may not match up with the situation described in the study. Averages can be confusing for both of these reasons.

What are the difficulties in making statistics understandable to patients?

MIKSAD: I like the framework that Dr. Talcott described. The researcher’s first task is to develop studies so that we can compare outcomes across all patients and obtain a bird’s-eye view of a particular problem. Then a lot of effort needs to be made to bring that information back down to the patient level so we tell a patient what his risk of incontinence is, for example.

An important part of the research is to make sure that the words used in the survey are understood by patients answering it. Some interesting work has been done on this, and it turns out that very common words used on surveys for men with prostate cancer aren’t understood by everyone — impotence, incontinence, even erection. And if the patients who are answering surveys don’t understand the questions they’re answering, the information is useless.

Making treatment decisions

What advice can you give readers who are contemplating various treatments for prostate cancer?

TALCOTT: Well, there’s a difference between the symptom itself and how the patient feels about it. Somebody is likely to report that a symptom is severe or frequent because they notice it and it bothers them, but there are some “disconnects,” like the patients with urinary incontinence described earlier. We do a pretty good job of describing the condition, but we don’t do a very good job of describing how patients will react to it. However, there are some things we can do, like explain exactly what we mean by incontinence. A few drops of urine might not seem like much, but if you talk to men about how it affects their lives, they’ll say that they worry about the odor, they worry about feeling unclean, they worry about embarrassing themselves, they worry about the logistics of changing their pads. So, one of the things that we can do is describe for them what’s likely to happen functionally and how other men have reacted to urinary incontinence.

It sounds like the horse is already out of the barn by the time patients get information on possible side effects. Can we get it to them sooner?

TALCOTT: You’re right. We need to walk through the options with them, in detail, before they undergo treatment. The problem is that this is a very complex disease, and there’s a lot of information that needs to get transmitted in a short period of time. An hour-long consultation in a clinic seems like a lot to most patients, but for a disease like this, it isn’t much. You have to convey the natural history of the disease, talk about the various treatments, discuss the likely complications, and, in the case of active surveillance, talk about the anxiety that some patients report when they don’t choose a definitive therapy. Obviously, the right time to give patients this kind of information is when they’re weighing the options. And we need to give them adequate time to make the decision. This is a decision that they’re likely going to have to live with for many years, because 95% of patients with localized prostate cancer live at least 10 years or die of something else.

Doctors generally don’t have the time they need to go through all of this with patients. What should patients do?

MIKSAD: I would encourage patients to help their physicians understand their lives. Given patients’ activities and values, what side effects and outcomes would be more of a problem for them and which would be less of a problem?

McCAFFREY: Patients should talk to other patients, perhaps in a setting where more than one patient is there. Perhaps there’s a professional there, maybe even a physician. Some recently diagnosed patients will say, “I just want the cancer out. I don’t care about the side effects, just get it out.” If problems crop up afterward, some patients don’t care, but others will say, “Gee, I wish I had known.” Talking with other patients might not change their minds, but at least they’ll feel more comfortable with their decision.

Are you advocating that newly diagnosed patients attend a support group?

McCAFFREY: Absolutely. But, I think it needs to be run by professionals, so that there’s someone who’s moderating the discussion. It’s overwhelming for someone who is trying to make a decision if one person is ranting and raving about an awful experience. Support groups should include people with a range of experiences — good outcomes and bad outcomes with different treatments — and professionals who can put it all into context.

One of the other things I’d like to say is that physicians should encourage patients to come to them with issues like incontinence and sexual dysfunction before they happen. Say, “If these things are a problem, come to me and I will help you find ways to deal with them.” I think a lot of men feel that they can’t go back to the person who did the procedure to get guidance.

Is that because patients don’t want to tell their physician what’s really going on?

McCAFFREY: That could be part of it. Or they may get a subtle message from the physician that it’s not something he or she knows how to fix. They may feel that if they bring it up that they aren’t going to get much help.

We’ve heard from patients who’ve said that their physicians seemed like they didn’t want to be bothered with problems like incontinence.

McCAFFREY: I won’t say that’s common, but it’s not unusual. I think the physician needs to say, “If this is a problem that you’re dealing with, come to me.” That physician doesn’t necessarily need to be the one to work on it, but he or she should have names and contact information for people who can help.

TALCOTT: I think that’s all sound advice. I think the best thing we can do is assure patients that they have time to make a decision that they’re comfortable with. Their cancer probably isn’t going to change significantly in the next few weeks or even months. They need to take the time they need to come to a decision that they’re comfortable with.

Given the nature of the disease, I would encourage patients to make the decision with their partner. It really affects the relationship. They should try to spend some time thinking through the decisions, understanding what the implications might be and getting a feel for them, and then make a choice with their partner.

What independent sources of information can a patient turn to for answers? How can patients determine whether the information they receive is accurate?

TALCOTT: When patients meet with a health care provider who quotes statistics on complication rates, they should ask questions like: What’s the basis for that information? Have you surveyed your own patients? Is that coming from an independent source that’s been evaluated? Are these your results or the results of another provider who you feel is your peer?

I can tell you that that almost never happens. I certainly know physicians who feel that all their patients’ outcomes are above average, like children at Lake Wobegon. That may be true, but the results need to be documented.

What tends to happen is the best reported results are given to patients because they sound good to everybody, but it’s a disservice to patients. Published data can give you a pretty good idea of what the results are likely to be for average patients who receive treatment from reasonably well-trained physicians. But my colleagues and I try to assess the likelihood of preserving or losing function.

Have any studies examined the differences between what physicians report and what patients really experience?

TALCOTT: About 15 or 20 years ago, we began asking patients to report on their condition at regular intervals, starting before they received treatment. We studied patients seen at the major Harvard teaching hospitals, even if they weren’t treated there. (Other researchers have asked patients to report on their condition, but their studies have had limitations. For example, they either hadn’t collected pretreatment data, included only patients who were 65 or older, or hadn’t focused on referral centers, where patients see specialists.) We found that patients reported a lot more problems than the treating physicians thought they had, even at excellent hospitals with experienced doctors. So the differences couldn’t be blamed on substandard care. [ See “Patient-reported outcomes,” below,  for a list of studies that have assessed side effects.]

How do you explain that discrepancy?

TALCOTT: It’s a combination of patients being reluctant to report bad outcomes, especially about embarrassing problems, to their physicians, although they will certainly share the details with their family, nurses, and other patients, and on paper questionnaires. And physicians focus on the positive, because that’s what we do when we deal with cancer. It’s a bit of a conspiracy by well-meaning participants: the physicians get patients through a difficult time, and the patients soldier on without affronting or insulting the doctors who did their very best.

McCAFFREY: I think that’s true. There are patients who want to be a “good” patient and not bring the problem to the doctor. The result can be an underreporting of problems.

TALCOTT: Physicians trying to understand the effects of their treatment also need to make sure that they assess what the patient’s function is like before treatment. Urinary incontinence and bowel problems aren’t very common before treatment for prostate cancer. Urinary obstruction is common in patients with benign prostatic hyperplasia, or BPH. Erectile dysfunction is highly correlated with age.

Measuring success of treatment

How is survival measured? What’s the difference between overall survival and progression-free survival? How does “survival” influence decision-making?

MIKSAD: Overall survival tends to be the gold standard of efficacy in oncology, but other endpoints, such as progression-free survival, have been used. Progression-free survival is the time from treatment until the cancer comes back, or until the patient dies from another cause. While these terms sound similar, there are some big differences. For example, just because progression-free survival might be significantly better with one treatment, that doesn’t mean that overall survival is significantly better, too. Some people might suggest progression-free survival is a more telling outcome because we would rather be without cancer longer. Other people say that it doesn’t really matter if the cancer comes back because all that really matters is how long they’re alive.

What are patients more interested in: being potent and continent or living longer?

McCAFFREY: It depends on the individual and what his lifestyle was like before he was diagnosed. For a man in his 70s who has been in a long-term relationship and doesn’t have a lot of sexual activity, long-term survival may be more important. Other men will focus on quality of life, incontinence, and sexual function. So much can depend upon the phase of life they’re in. Are they reasonably young, say in their 50s with young children? Are they older and experiencing changes in function already anyway?

MIKSAD: If patients have seen someone go through prostate cancer treatment and have side effects, it can affect what they value at the outset. But their values can change during treatment. It can be difficult for patients to accept that they made a choice in the past based on values they no longer have.

How can someone evaluate the success of different treatments?

TALCOTT: One of the biggest problems in evaluating the results of prostate cancer treatments is that there are almost no randomized trials comparing treatment modalities, and what evidence exists doesn’t indicate that one is better than the others. They differ more in terms of side effects than effectiveness.

In practice, we have an informal division of patients, with younger, healthier patients usually getting surgery and older patients, particularly those with more advanced disease, generally getting radiation. The surgery patients get a head start, so to speak, because their disease hasn’t advanced much yet, whereas the radiation patients’ therapy starts later in the course of the disease, when the cancer is more likely to have spread microscopically. As a result, they’re more likely to have a rising PSA after treatment, indicating that radiation didn’t get all of the cancer. They’re going to do poorly in terms of overall mortality, not just because they’re older and more likely to get sick from other diseases, but because they tend to have more advanced cancer to begin with. I think that’s confused patients — and well-meaning providers. Surgeons are right; their patients do tend to live longer, but that’s not necessarily because of anything the surgeons are doing.

MIKSAD: While we’re on the subject of bias, I should point out that any time we look at survey results, some bias is going to creep in, even with the best-designed studies. For example, patients who answer surveys are less likely to have significant health problems than those who don’t. Similarly, people who complete a survey on the Internet may not accurately represent the entire population of prostate cancer patients.

TALCOTT: And those biases tend to really kick in during studies of patients with advanced disease. When they don’t respond, it’s often because they’re too sick or because they’ve died. For patients with early prostate cancer, we see a lot less bias in terms of the response rates.

We haven’t talked much about impotence. Any comments on that?

TALCOTT: Certainly men have the expectation that they may lose potency after treatment, but the impact of impotence will vary depending on their relationships. Men in a stable relationship may find that impotence affects their sexuality a lot less than they thought it would. This might be surprising to some people, but there are a lot of ways to engage in sexual activity without vaginal penetration. I’ve had more than a few patients in long-lasting relationships say that reinventing their sexual relationship resulted in a more pleasurable exchange. On the other hand, for younger men or for men who have been recently divorced or widowed, sexual function may be more important, at least in the short run.

There’s a lot of individual identity tied to sexual functioning, so impotence may mean more than simply the inability to get an erection sufficiently rigid for intercourse. I think it’s fair to counsel patients that erectile dysfunction may have meanings that go beyond their role in a sexual relationship.

McCAFFREY: Occasionally I run into men who say, “My sex life is over.” But there are steps men can take, such as using a vacuum pump, oral medication, or injections, to have an erection. Some men don’t know these things exist and resign themselves to thinking that nothing can be done. That’s sad because for many men, self-identity and self-worth are linked to potency.

TALCOTT: This is where a conversation between the patient and his partner is key. They need to have an open discussion without any presuppositions. At first, this can be difficult because people generally don’t talk openly about sexual functioning. But once you start to talk, you can learn an awful lot that can be helpful in making decisions, and when you look back on the conversation, assumptions that you may have made beforehand may not prove valid.

Are there questionnaires that might help patients figure out what complications would be most bothersome to them?

TALCOTT: One of the problems with trying to predict how something might affect future quality of life is that people adapt. It’s certainly a good mental exercise, and I encourage patients to consider how they would feel about the various possible complications, but it can be very hard to predict how things will play out. For example, if you asked someone how their quality of life might change if he or she were blind, you would get a result that indicates a pretty spectacular decline. If you asked a blind person how much their blindness affects their quality of life, you’d probably get a much different answer. People develop strategies for overcoming dysfunction, making it hard to predict how they’ll feel down the road.

Any parting comments or final words of advice?

MIKSAD: I think outcomes research will continue to improve decision-making. I encourage patients to discuss the kinds of issues that we’ve brought up here so that they can get the information they need to make decisions that are right for them. And I encourage them to participate in outcomes research studies: complete questionnaires or answer the phone when a surveyor calls.

TALCOTT: I’d echo that. As we come to appreciate that patient-centered care isn’t just a slogan, outcomes research is going to be increasingly important. We need to put more effort into understanding what patients’ needs are and listening to their voice. That will help us do a much better job at providing health care that patients need rather than what we assume they need.

McCAFFREY: Contact others who have been where you are. Ask about side effects, how they coped and adapted. You might learn about something that you hadn’t considered that turns out to be the best choice for you.

Originally published Jan. 1, 2008; Last reviewed April 20, 2011