2014 Spirulina randomized self-experiment

Spirulina users’ randomized trial on self-rated allergy symptoms: null result.
psychology, experiments, statistics, nootropics, R
2014-10-052014-11-27 finished certainty: unlikely importance: 0


The sup­ple­ment Spir­ulina has been sug­gested to help al­lergy symp­toms. A ran­dom­ized self­-ex­per­i­ment is run by scead­uwe April—Au­gust 2014. Analy­sis sug­gests no effect of the spir­uli­na.

is a pho­to­syn­thetic al­gae which can be grown for food (dried spir­ulina is high in pro­tein); it is also used by some as a sup­ple­ment due to var­i­ous other sub­stances in it. There is (Mao et al 2005, Cingi et al 2008) sug­gest­ing it may help al­lergy prob­lems like hay fever, but noth­ing con­clu­sive.

Background

scead­uwe suffers from al­ler­gies and post-nasal drip each year dur­ing pollen sea­son, so he be­came in­ter­ested in whether spir­ulina might help and re­duce the need for his an­ti-al­lergy med­ica­tion (/“Zyrtec”), but he did­n’t want to waste money on a dodgy sup­ple­ment. I gave him ad­vice on de­sign­ing & run­ning a self­-ex­per­i­ment test­ing spir­uli­na, and offered to an­a­lyze the data should he ever fin­ish the ex­per­i­ment.

Experiment

He pro­posed do­ing a re­peat­ed-mea­sures n = 1 blind ran­dom­ized self­-ex­per­i­ment dur­ing spring/­sum­mer 2014, in which he would rate post-nasal drip each day on a scale 1-5 where 1=­com­plete­ly-clear and 5=­can’t-breath. The spir­ulina tablet would be put into pills with the placebo pills be­ing filled with (safe, cheap, sim­i­lar weight, pos­si­bly help­ful for other things), for doses roughly around 0.8g of cre­a­tine / 0.7g of spir­uli­na; these pills could be dou­bled for some blocks if he chose. The cited ex­per­i­ments used long blocks, so he’d ran­dom­ize mul­ti­ple weeks in pairs of blocks 50:50, and the blind­ing/ran­dom­iza­tion would be done with my usual trick of two iden­ti­cal but marked con­tain­ers. Data col­lec­tion was lim­ited by the sea­son­al­ity of al­lergies: once the worst days pass, no more data can be col­lected un­til the next year.

As part of the base­line, he took cre­a­tine for some days be­fore be­gin­ning the spir­ulina ex­per­i­ment proper from 2014-04-04 to 2014-08-08, with a long lapse in June/July when he be­came dis­tracted by other mat­ters By 8 April, it had be­come clear the blind­ing had failed (“spir­ulina burps”), so it was just ran­dom­ized. And at the end, scead­uwe be­lieved the spir­ulina was in­effec­tive and turned his data over to me for analy­sis.

Data

There were sev­eral vari­ables to code up:

  1. the post-nasal drip / al­lergy self­-rat­ing (1-5); the out­come vari­able

  2. whether it was pollen sea­son (April to May, rough­ly)

  3. spir­ulina dose

  4. whether he used an­ti-al­lergy med­i­cine on the worst days

  5. cre­a­tine dose

  6. where he was liv­ing

    scead­uwe moved apart­ments in Au­gust 2014, and I thought this co­vari­ate was worth in­clu­sion as differ­ent room­s/lo­ca­tions may ag­gra­vate al­ler­gies differ­ent­ly.

Due to the lapse and in­clud­ing the base­line, that yields 95 days of da­ta.

De­scrip­tive:

spirulina <- read.csv("https://www.gwern.net/docs/nootropics/2014-sceaduwe-spirulina.csv",
    colClasses=c("Date", "logical", "integer", "integer", "logical", "logical", "integer", "factor"))
summary(spirulina)
##      Date            Pollen.season   Allergy.rating      Spirulina        Spirulina.random
## Min.   :2014-02-28   Mode :logical   Min.   :2.00000   Min.   :0.000000   Mode :logical
## 1st Qu.:2014-03-25   FALSE:79        1st Qu.:3.00000   1st Qu.:0.000000   FALSE:33
## Median :2014-05-04   TRUE :16        Median :3.00000   Median :0.000000   TRUE :62
## Mean   :2014-05-12   NA's :0         Mean   :2.95789   Mean   :0.505263   NA's :0
## 3rd Qu.:2014-07-13                   3rd Qu.:3.00000   3rd Qu.:1.000000
## Max.   :2014-08-09                   Max.   :5.00000   Max.   :2.000000
## Allergy.medicine    Creatine        Location
## Mode :logical    Min.   :0.000000    A:90
## FALSE:86         1st Qu.:1.000000    B: 5
## TRUE :9          Median :1.000000
## NA's :0          Mean   :0.957895
##                  3rd Qu.:1.000000
##                  Max.   :2.000000
library(ggplot2)
qplot(Date, Allergy.rating, color=as.ordered(Spirulina), data=spirulina) +
 theme_bw() +
 xlab("Date (2014)") + ylab("Allergy self-rating") +
 theme(legend.title=element_blank()) +
 geom_point(size=I(5)) +
 stat_smooth()
Al­lergy self­-rat­ing over time, col­ored by spir­ulina dose (0, 1, 2 pill­s), and smoothed

Analysis

At a first stab, I load up the data and regress on #2-6:

l <- lm(Allergy.rating ~ Pollen.season + Creatine + Location + Allergy.medicine + Spirulina,
        data=spirulina[spirulina$Spirulina.random,])
summary(l)
##
## ...Coefficients:
##                        Estimate Std. Error  t value   Pr(>|t|)
## (Intercept)           3.7497249  0.4032818  9.29803 5.9712e-13
## Pollen.seasonTRUE     0.0995488  0.2732337  0.36434  0.7169796
## Creatine             -0.5534339  0.2489801 -2.22280  0.0302834
## Location B           -1.1608519  0.3648258 -3.18193  0.0023872
## Allergy.medicineTRUE -0.5023112  0.3210568 -1.56456  0.1233211
## Spirulina            -0.01277196  0.1390217 -0.12746  0.8990337
##
## Residual standard error: 0.631029 on 56 degrees of freedom
## Multiple R-squared:  0.250248,   Adjusted R-squared:  0.183305
## F-statistic: 3.73827 on 5 and 56 DF,  p-value: 0.00544645

library(MASS)
pl <- polr(as.ordered(Allergy.rating) ~ Pollen.season + Creatine + Location + Allergy.medicine + Spirulina,
           data=spirulina[spirulina$Spirulina.random,])
summary(pl)
## ...Coefficients:
##                            Value Std. Error    t value
## Pollen.seasonTRUE      0.5572250   0.865166  0.6440669
## Creatine              -1.8439587   0.846798 -2.1775671
## Location B           -16.4522527 634.533164 -0.0259281
## Allergy.medicineTRUE  -1.8680170   1.078616 -1.7318653
## Spirulina              0.0466616   0.445739  0.1046836
##
## Intercepts:
##     Value      Std. Error t value
## 2|3  -3.910809   1.418710  -2.756594
## 3|4  -0.672510   1.281926  -0.524609
## 4|5   1.977223   1.601942   1.234267
##
## Residual Deviance: 106.639735
## AIC: 122.639735

The im­me­di­ate an­swer to the pri­mary causal ques­tion is clear: no, the spir­ulina has no ap­par­ent effect ei­ther good or bad.

The other vari­ables are in­ter­est­ing, though: of course the al­lergy med­i­cine has an effect (pre­sum­ably that was proven by the clin­i­cal tri­als that got it ap­proved), but what’s cre­a­tine do­ing with p < 0.03 and Lo­ca­tion at p < 0.002?

Look­ing at Lo­ca­tion closer, it only changes right be­fore the data se­ries ends in Au­gust, and Au­gust should be well past the pollen dan­ger sea­son, so it may be a con­found with al­ler­gies over the sea­sons, which we’d ex­pect to look some­thing like an in­verted U-curve peak­ing in spring/­sum­mer. This sug­gests adding a qua­dratic term to the re­gres­sion mod­el, some­thing like I(as.integer(Date)^2), but then a step() (which pe­nal­izes com­plex­i­ty) prefers us­ing the Lo­ca­tion vari­able to the qua­drat­ic:

l2 <- lm(Allergy.rating ~ Date + I(as.integer(Date)^2) + Pollen.season + Creatine + Allergy.medicine + Spirulina + Location,
         data=spirulina[spirulina$Spirulina.random,])
summary(l2)
## ...Coefficients:
##                           Estimate   Std. Error  t value  Pr(>|t|)
## (Intercept)            2.65011e+04  4.87941e+04  0.54312 0.5892816
## Date                  -3.26909e+00  6.01720e+00 -0.54329 0.5891648
## I(as.integer(Date)^2)  1.00832e-04  1.85509e-04  0.54354 0.5889936
## Pollen.seasonTRUE      2.12637e-01  3.34582e-01  0.63553 0.5277659
## Creatine              -8.82748e-01  5.25005e-01 -1.68141 0.0984595
## Allergy.medicineTRUE  -4.49432e-01  3.33177e-01 -1.34893 0.1829898
## Spirulina             -2.31188e-01  3.59458e-01 -0.64316 0.5228438
## Location B            -1.34235e+00  4.47436e-01 -3.00010 0.0040776
##
## Residual standard error: 0.639404 on 54 degrees of freedom
## Multiple R-squared:  0.257707,   Adjusted R-squared:  0.161484
## F-statistic: 2.67822 on 7 and 54 DF,  p-value: 0.0186685
step(l2)
## ...Call:
## lm(formula = Allergy.rating ~ Creatine + Allergy.medicine + Location,
##     data = spirulina[spirulina$Spirulina.random, ])
##
## Residuals:
##       Min        1Q    Median        3Q       Max
## -1.200000 -0.200000 -0.200000  0.323413  1.800000
##
## Coefficients:
##                       Estimate Std. Error  t value   Pr(>|t|)
## (Intercept)           3.757143   0.267633 14.03840 < 2.22e-16
## Creatine             -0.557143   0.196389 -2.83693  0.0062644
## Allergy.medicineTRUE -0.422222   0.232107 -1.81908  0.0740620
## Location B           -1.200000   0.327782 -3.66096  0.0005447
##
## Residual standard error: 0.621037 on 58 degrees of freedom
## Multiple R-squared:  0.247869,   Adjusted R-squared:  0.208965
## F-statistic:  6.3714 on 3 and 58 DF,  p-value: 0.00083053

My guess here is that there’s not enough data, par­tic­u­larly in the June/July gap, to jus­tify the curve when lo­ca­tion A vs lo­ca­tion B hap­pens to over­fit the lat­ter points so well, but I still be­lieve this Lo­ca­tion B es­ti­mate is be­ing dri­ven by sea­sons.

What about cre­atine, is that sea­sonal too? scead­uwe did­n’t do cre­a­tine ini­tial­ly, but he started cre­a­tine be­fore the pollen sea­son and never stopped in the data, so it should­n’t be dri­ven by the same end-of-al­ler­gy-sea­son effect. A plot:

qplot(Date, Allergy.rating, color=as.ordered(Creatine), data=spirulina) +
 theme_bw() +
 xlab("Date (2014)") + ylab("Allergy self-rating") +
 theme(legend.title=element_blank()) +
 geom_point(size=I(5)) +
 stat_smooth()
Al­lergy self­-rat­ing over time, col­ored by cre­a­tine dose (0, 1, 2 pill­s), and smoothed

The cre­a­tine es­ti­mate seems to be dri­ven by the pe­riod with 2 pills. I don’t know of any prior rea­son to ex­pect cre­a­tine to have any effect on al­ler­gies or post-nasal drip, so I won­der if this turned out to be a sub­tler form of the sea­sonal effect due to the ex­tra weight­ing of the dou­ble-dose in the re­gres­sion?

If I ex­pand the dataset to in­clude the base­line as well, and I look at cre­a­tine as a fac­tor, the effect also seems to dis­ap­pear:

l3 <- lm(Allergy.rating ~ Date + I(as.integer(Date)^2) + Pollen.season + as.factor(Creatine) + Allergy.medicine + Spirulina + Location,
            data=spirulina)
summary(l3)
## ...Coefficients:
##                           Estimate   Std. Error  t value  Pr(>|t|)
## (Intercept)            9.07174e+03  1.64581e+04  0.55120 0.5829234
## Date                  -1.12123e+00  2.03062e+00 -0.55216 0.5822707
## I(as.integer(Date)^2)  3.46557e-05  6.26343e-05  0.55330 0.5814923
## Pollen.seasonTRUE      1.77142e-01  2.93014e-01  0.60455 0.5470693
## as.factor(Creatine)1   3.44745e-01  3.10455e-01  1.11045 0.2699001
## as.factor(Creatine)2  -4.80779e-01  4.20889e-01 -1.14229 0.2565024
## Allergy.medicineTRUE  -4.44689e-01  3.22589e-01 -1.37850 0.1716244
## Spirulina             -1.65976e-01  1.55833e-01 -1.06509 0.2898163
## Location B            -1.15120e+00  3.89378e-01 -2.95650 0.0040155
summary(step(l3))
## ...Coefficients:
##                       Estimate Std. Error  t value   Pr(>|t|)
## (Intercept)           2.888889   0.146343 19.74048 < 2.22e-16
## as.factor(Creatine)1  0.294785   0.171125  1.72263  0.0883899
## as.factor(Creatine)2 -0.246032   0.221250 -1.11201  0.2690973
## Allergy.medicineTRUE -0.405896   0.225167 -1.80265  0.0747911
## Location B           -0.983673   0.291490 -3.37464  0.0010922
##
## Residual standard error: 0.620883 on 90 degrees of freedom
## Multiple R-squared:  0.170613,   Adjusted R-squared:  0.133752
## F-statistic: 4.62848 on 4 and 90 DF,  p-value: 0.00191632

If cre­a­tine were help­ing with al­lergies, one would ex­pect the effect to be at least mo­not­o­n­ic, what­ever the de­tails of the dose-re­sponse curve; but in­stead we see that 1 pill of cre­a­tine cor­re­lates with in­creases in al­ler­gies and 2 pills with de­crease! So my best guess is that this is spu­ri­ous too like the Lo­ca­tion es­ti­mate.

Improvements

If one were try­ing to run such an ex­per­i­ment, there seem like some ways to im­prove on this:

  1. fix the blind­ing some­how; per­haps add some strong-tast­ing in­gre­di­ent to the cap­sules?

  2. more sys­tem­atic data; the gap in June/July ham­pers any at­tempt to fit a curve to al­ler­gies

  3. find some way to pre­dict how bad al­ler­gies will be on each day: there must be lo­cal weather data on pollen count, or maybe par­tic­u­late count, or fail­ing that, per­haps some small afford­able de­vice for mea­sur­ing air qual­i­ty.

    Is it affected by other things like hu­mid­ity or tem­per­a­ture? That would be good to in­clude.

  4. more di­rectly mea­sure al­lergies: per­haps quan­tify it some­how like num­ber of sneezes or kleenexes used? Since al­ler­gies are re­lated to his­t­a­mi­nes, per­haps some chem­i­cal level could be mea­sured in sali­va? Or one could go in­di­rect: bad al­ler­gies might in­ter­fere with think­ing and so a bat­tery of cog­ni­tive tests could ex­pose hid­den changes.

Conclusion

Spir­ulina did not cause a large de­crease in al­lergy symp­toms, al­though this re­sult is weak­ened by rel­a­tively small sam­ple size & ap­par­ent tem­po­ral/sea­sonal changes. (These is­sues could be im­proved in fu­ture ex­per­i­ments by more sys­tem­atic data col­lec­tion and longer self­-ex­per­i­ments.) Two other vari­ables are in­ter­est­ingly cor­re­lated with de­creas­es, but on closer in­ves­ti­ga­tion, both seem like they may be prox­y­ing for sea­sonal effects and un­likely to be caus­ing de­creas­es.

I would not sug­gest scead­uwe sup­ple­ment spir­ulina un­less he has some other rea­son to use it.

See Also