Site map

 

Discovering what treatments for atrial fibrillation  will work for you:

Evaluating research reports and experimenting with supplements

 

If you want to skip the statistical and other background stuff,  you can miss the parts in the lighter-colored boxes

 

There are two kinds of research: outcome and process.

 

Outcome research and the placebo effect

 

In outcome research, the investigator is trying to determine whether applying a treatment results in a different outcome than not using the treatment or in using some other treatment. Understanding how the treatment works is not the main goal of this kind of study, only whether it leads to the desired outcome. In many cases, knowledge about process comes later, after studies have been done that show that it works. Research on aspirin is a good example of this sequence.

 

This is a good place to talk about the placebo effect, which comes from factors that are present to some degree in all or in many treatments and which can result in improvement regardless of the specifics of the treatment.

 

A very important placebo effect is the degree of belief the subject (the “S”) has in his treatment and the degree of hope this belief engenders. Another factor can be the positive attention the S receives for being in the study or for being a cared for as a patient. Another is the diversion from pain or other distress created by the treatment. So, hope, diversion and positive attention can by themselves lead to increasing levels of physiological, psychological, and observational measures of treatment success.

 

Outcome studies reported in peer-reviewed journals will have a placebo group; that is, a group which receives a fake treatment that is in every way similar to the real one except that it lacks the chemical or other essential ingredient.  

 

The placebo group for a test of a medication might receive an amount of attention equal to that of the medication group, and get a “sugar pill” that in looks and tastes like the medication except for its chemical composition. The goal is to remove the placebo-effect component from the effect of the treatment; so that, if the treatment group is significantly different than the placebo group, the investigator can say that it was specific treatment that is causing (or is correlated with) the effect, not placebo features that might be present in many other treatments.

 

The study will also be “double-blind” meaning that neither the S nor the person administering the real or fake treatment knows which is which. The assumption is that the S's knowledge of which treatment he is receiving will affect his outcome via the belief component of the placebo effect. If the person administering the real or fake treatment knows which is which, he may influence the subject’s belief in whether his treatment will work for him.   

 

So, what does this mean to you? It is important to realize that:¨1) the placebo effect is not necessarily bad from a patient’s standpoint – after all, every little bit helps--, but that, 2) its effects may be limited and may disappear over time. Then, you will need to look for another placebo treatment, and then another, etc.. So it may be a “real” treatment will work better.  But, a pure placebo can have considerable power. You may have read that the placebo effect has effects on the brain that can mirror those of a real treatment.

 

Reading posts on internet bulletin boards that describe positive experiences with a certain treatment can create a kind of placebo effect.  These are the testimonials that make us hopeful, thinking, “This will work for me!!!” in spite of the fact that there is no scientific evidence that the treatment is effective for the particular group of people of which you are a member. Belief and hope feel good, and, as implied above, can have real physiological effects. (In fact, I believe there was a study that showed that even if a research S is told he is being given a placebo such as a sugar pill, the placebo treatment can still have an effect!) 

 

My own opinion is that, most of the time,  you can do better than to use treatments that depend solely or mostly on the placebo effect for their power. Do your homework, and find a treatment that works for people like you because of its unique features. You will get enthusiastic and hopeful about it and you will get a chemical or other effect.  You will have the best of both worlds.

 

Process research

 

In process research, the investigator is trying to determine how a treatment works to create its effect. Being able to describe the process will certainly lead to predictions of outcome. It will make a big difference when the treatment has an effect on a part of the brain that is known to be involved in the condition being treated; or, conversely, if the treatment substance turns out to be  toxic to the liver or heart.  Most tests of medications begin with process research that must first look promising before first doing small-scale studies and then finally taking on the huge expenditure of time and money required by large-scale outcome research.

 

Sometimes, outcome studies of a treatment whose process looks promising are disappointing, perhaps because some part of the what was going on was not discovered by the initial research on the process.

 

Many – but not all – of the suggestions for the use of supplements for various conditions including AF are based on process research, or on conclusions about process deduced from a variety of sources. Unfortunately, the outcome research needed to back up process assumptions is often lacking, and a small series of individual positive or negative experiences are a poor substitute for such research.

 

Conclusions from a small sample must be very tentative: The reported effect is unreliable and you cannot expect that it will be replicated when other similar-sized groups are tested. Also, when sample sizes are very small, and the spread of scores around the means of the two groups is often not reported. The reason the latter is important is because large spreads mean that there is probably a lot of overlap between the groups, so that the differences in the means become meaningless.

 

This does not mean that exploratory or small-sample research is without value for the individual. It does mean that more must be done in terms of examining the data and in doing additional research before any conclusions can be drawn.

 

There are several things an individual can do when faced with a treatment he feels might benefit him on the basis of process research or from testimonials (case histories). He can look carefully at the process research and examine the believability of testimonials. He can take special care in setting up a valid “personal experiment” as below.

 

He should also look especially carefully at  the benefit/cost ratio. The benefits are the positive effects of the treatment;  the costs include: side effects; the expense, time and effort that must be expended, and the consequences of failure.

 

The last will vary widely. It may not matter much if a supplement fails to work. It will matter more if removing all the fillings in your mouth does nothing. It will matter a great deal if replacing warfarin with nattokinase turns out badly. This is not to say that warfarin does not have its costs, but there are many outcome studies to support both its efficacy and to quantify its dangers.

 

From our point of view as consumers, we’d like to see that the treatment works for Ss who are similar to us (outcome research), but we would also like to be reassured in ways that only process research can. We want to know that researchers understand how the treatment works so that they can assure us that it is not harmful in ways that might be discovered after a long period of time. Knowing the process completes our judgment that the treatment is a good one. We learn that it is successful from outcome studies and we know why it is successful from process research.

 

Of course, if we were Bill Gates, we would go to the limit. We would have an army of scientists and doctors making frequent exact measurements of the effects of the process on our unique body chemistry so that we would know that the treatment was working as expected in our individual case.

 

Statistics

 

As a medical consumer, you are not only interested in finding a treatment that works, but one that works better than any other treatment. Research comparing treatments is difficult to do because you need a sufficient number of Ss in each group.

 

Statistics reported in the "Results" section of a journal article as well as conclusions based on these Results that are reported in the "Conclusions" section can be misleading when you try to apply them to an individual such as yourself.

 

The “p < .xx (that is, p is less than some decimal fraction)"  means that the probability is less than the number indicated that the difference is the result of chance. But the fact that an effect is “real” does not mean that it is clinically significant.  For example, statistically significant results are especially likely when sample sizes are very large, even though the results when the effects of sample size are controlled may be more modest.

 

Let's say that  the risk of stroke in a population of ill people is 4%, and a treatment reduces the risk to 3%. The results can then be publicized as showing the treatment “reduced the risk of stroke by 25%”. This sounds impressive, but is this 1% reduction worth the cost of the treatment and its unpleasant, dangerous or unknown side effects?

 

Another statistic that it often misinterpreted is the correlation coefficient. A high correlation between two variables means that when one variable changes, the other tends to change along with it. A correlation can be statistically significant (different from 0) but its clinical significance can be nil because there is so little actual covariance.

 

And correlation says nothing about cause and effect. Two variables may covary because each one is related to a third variable. [ I cannot think of a good example in the area of AF. Perhaps you can? ] Anyway, the point is that a correlation has to be further analyzed using process-  or other research if assertions about cause and effect are going to be made. 

 

Process and outcome research in action

 

When you go to a site like rxlist, that provides information on various medications, you will find sections on Clinical Pharmacology (the findings of process research) and on Indications and Usage (outcome research). When you explore treatments for AF, you will be looking at statements about

 

1) ...the process (Does that medication seem as though that would work? Does the rationale for predicting its success make sense?);    and,

                                         

2)... the outcome; that is, success rates --- for different types of AF, measured at different times after the procedure, and for different methods of measuring success.

 

The bottom line

 

When looking over the results of research, you can ask these questions:

 

  • Are the research Ss similar to me in ways that mean I can apply the results to my case? Is there a group of research Ss that has the same type and duration of AF as mine? Is this group of similar age and do the Ss have the same risk factors as I do?

  • Is the size of the sample large enough so that generalizations can safely be made to the whole population of similar Ss? (If  the sample size is small, this will usually be mentioned in the Discussion section of the research report.) 

  • Is the result clinically significant? A treatment can make a statistically significant difference, in that a similar result based on chance would occur very rarely in samples of similar size. A clinically significant treatment will suggest that it may make the kind of difference you want it to make in your life. Reports that say that a treatment will increase or decrease a factor by what seems like a significant percentage when the percentage it is increasing or decreasing is very low to begin with are misleading (see above).  

  • What are the benefits and costs (side effects; financial expense, time and effort required, consequences of failure) of this treatment? 

  • How does this treatment stack up against other treatments? If the tests of each treatment are based on small sample sizes and or there is a lot of variation around the means of each treatment group, differences between the means of the treatment groups may not mean anything.

This may well be the case in comparisons between treatments or doctors for AF: For example, there may not be any real difference between reported success rates of 80% vs. 90+% for different  surgical or ablation treatments for AF.  You have to look at the data and perform the appropriate statistics before drawing any conclusions about the significance of the difference between means. For example, a rate that seems high but is based on a definition of success that includes patients who are still on medication, or whose freedom from AF has been self-reported may not really be higher than a procedure whose rate is based on less favorable measures.

 

Positive results of process-, case history- or outcome research may lead you explore or undergo  a promising treatment. But research can also encourage you to consider waiting until better treatments are available. (People in continuous AF may not want to wait because of the development of remodeling or of other factors that  reduce the probability of success for people with chronic AF.

 

Doing your own personal experiment

 

Because treatments for AF are constantly changing  and new ones are continually emerging, there may not be enough research to indicate whether or not the most promising treatments -- promising on the basis of process analysis-- will actually be successful. But you may not want to wait a year or more for outcome research to catch up. .And even if there is lots of research, the final test must be whether it works for you.

 

So, ideally, you must try it out on yourself and see what happens, and if it doesn't work, try something else. You cannot do this with surgery or ablation or course because your first procedure will place limits on what treatments can follow. And you can't administer and withdraw surgery or catheter ablation the way you can other treatments. But you can try as many supplements or medications as you like in all kinds of combinations.

 

Sometimes the supplement or medication is so powerful that it can cut through the effects of other variables in your body or your environment with a definite positive result.  A negative result is usually not so clear cut, because there are usually can be many reasons why the treatment didn’t work besides the fact that the treatment has no effect whatsoever under any circumstances.

 

Doing an experiment to determine whether or not a supplement will work for you is not necessarily a simple matter. You may be persuaded that the treatment is doing something for you, but would someone else be convinced?

 

For example, it is easy to get involved a behavior called “superstitious pyramiding”, which can occur when person who is desperate for results tries a series of different supplements. The person starts with one supplement, which may work for a while, perhaps because he hopes it will. The person then tries another supplement he has read about, but he keeps on taking the old supplement “just in case it might be doing some good”.  He does not want to take the chance of having a bad day or of having his condition return. In this way he can end up taking quite a number of supplements that may not been working since their initial placebo effect wore off.

 

So, here is what you must do try to do when doing your own personal experiment: 

  • You must deal with other variables that might affect the results. For example, you must stop all other activities that might the effect you are trying for, or you must hold their levels constant. 

  • The other thing you must do is to take the treatment, and  you must stop taking it and see if the improvement or other effects disappear. You will understand that if there is no difference when you stop the treatment, it is likely that something other than the treatment is causing the improvement – unless the application of the treatment resulted in a permanent change that would endure when the treatment stops. 

  • Try to quantify ratings of your behavior or of the way you feel by using rating scales.

  • Choose a success rating or ratings that is meaningful to you: # of episodes of AF; intensity of episodes of AF, side effects of supplementation; or, increased energy, reduced irritability, more sleep, etc..  

  • Keep a journal into which you enter these ratings and other indications of treatment effects every day. Graph the ratings you make when on and off supplementation. Be alert for side effects.

  • Have someone else observe your behavior. Do they notice difference in the way you are acting or appear to be feeling?

 

Here is a couple of web sites that will give you more information on single-case experimentation:

 

http://www.unlv.edu/faculty/pjones/singlecase/scsantro.htm

 

http://wps.ablongman.com/ab_leary_resmethod_4/0,7866,765537-,00.html

 

 

A final note to all of us who give advice based on research reports

 

We can start by indicating whether the research is anecdotal (based on non-expert reports of personal experience), case studies (expert analysis of a few cases), small-sample research (which almost by definition means that final word on any conclusions awaits the result of large sample research), or large-sample research

 

In the case of outcome studies, we should examine the statistics reported in the Results section and try to determine the clinical significance of the results. This is what people do who create reviews or conduct meta-analyses for publication. It's hard work!

 

If we are passing on the results of process research, we should try to determine the relevance of the research to the treatment situation of interest. For example, an experiment on a rat or dog on the effects of a supplement on a factor that may be distantly related to AF may not carry much weight.

 

If you are reporting individual experiences or a series of cases that might be the basis for a doctor’s recommendation, looking at the data is important as always. For individual experiences, this  means asking things like, “How well did the person do his personal experiment?" “Could the effects be mostly placebo effects?"

 

.In the case of a doctors recommendation or other conclusions based on a series of case studies: ”How many positive and negative instances are we talking about here?” “How is success measured, and could the reported outcomes be affected by a patient’s desire to please the doctor? "

 

We should also be sure to say if we are merely passing on second-hand reports of research, such as those presented by the media. Many times, a magazine, newspaper, or TV program reports as established fact the findings of small-sample research as though it had great clinical significance. Other media pick it up and it can soon become conventional wisdom, even though the research does not justify such a conclusion. 

 

So, discovering what will work for us or whether something is working in treating AF is not easy. There is so much information out there and so much that is advertised as being useful. And people are so different; it is likely that there are many approaches that work for at least a few people.

 

 What is lacking is measurement of the physiological variables in ourselves that affect the way we respond to different treatments. Routine assessment of our genetic makeup will be a step in this direction, as is the determination of the differences in the speed at which we metabolize different medications.

 

Site map Top