"Eat Chocolate - lose weight"
What a catchy headline, and even catchier diet concept.
Even better - its PROVEN in a clinical trial!
Journalist John Bohannon recently published a piece on the io9 blog about how he fooled millions into thinking chocolate helps weight loss.
I think that this article is a MUST read for biotech investors - especially those who are not very familiar with statistics.
Here's the short version
Bohannon and colleagues set up a clinical trial in Germany where 15 patients were randomly assigned to one of 3 diets (control, low carb, and low carb + daily chocolate).
After the 3-week trial, the team crunched the data looking for possible signals.
They found that the group that ate chocolate in addition to a low-carb diet lost weight 10% faster than the group who only did low carb dieting. - And get this, - the results was statistically significant!
It was also meaningless - and that is the whole point of the article.
The Takeaway for Biotech Investors
In this article Bohannon does a great job of explaining why statistical significance can be completely meaningless when it comes to clinical trials with small numbers of patients.
Here's the key as Bohannon explains:
If you measure a large number of things about a small number of people, you are almost guaranteed to get a statistically significant result.
Bohannon goes on to explain the problem of small trials:
Think of the measurements as lottery tickets. Each one has a small chance of paying off in the form of a `significant` result that we can spin a story around and sell to the media. The more tickets you buy, the more likely you are to win. We didn't know exactly what would pan out - the headline could have been that chocolate improves sleep or lowers blood pressure - but we knew our chances of getting at least one `statistically significant` result were pretty good.
And this is the key takeaway for biotech investors.
Too often in biotech we hear of (especially smaller) companies running small clinical trials and then announcing "positive" results that are statistically significant for some measure. But the trial was not sized properly to draw a meaningful conclusion.
The company's stock price gets bid up, investors assume that the next phase of clinical testing - involving up to 10 times as many patients - is a sure-thing.
When the larger trial fails the stock price comes crashing down.
This is one of the top RED FLAGS that I highlighted in part 2 of our free video series on Mastering Biotech Binary Events.
Small 'n' Huge Risk
Going from small trials to much larger trials caries a huge risk that the scientific thesis is simply invalid.
Recent examples that come to mind include Can-fite Biopharma (NASDAQ:CANF). Canfite ran a phase 2 trial in a small number of patients and concluded that their therapy shows promise in psoriasis.
When we looked at their trials, we suggested that investors would do well to avoid the stock because the data did not support a positive outcome in the larger trial.
Immunocellular Therapeutics (NASDAQ:IMUC) is another example that comes to mind.
Back in 2013 we pounded the table suggesting that their remarkable survival results form a tiny phase-1 study in brain cancer patients were caused by cherry picked patients rather than by the effect of their drug. When the larger phase 2 trial failed, the stock got crushed from over $4 per share to less than $1 per share.
Understanding the principles that Bohannon explains so well in his article can really help investors improve their returns from biotech binary event investments.
Here's a link to Bohannon's article - it's really worth a read.
