Newton et al. (in press)1 report on an empirical study that examined the impact of implanting acoustic tags up to 12.6% of fish body mass on the success rate of salmon smolt outmigration. Commonly, biologists favour the ‘2% rule of thumb’– meaning that transmitters should not exceed 2% of the total mass of the individual, so as not be a significant burden to the fish carrying the transmitter. This 2% rule has been under attack for some time, as supporting evidence for this rule is slight. Newton et al. cite multiple studies that have demonstrated impacts of transmitters on fish, as well as others failing to observe impacts.
Given that outmigration by salmon smolts is a key life history challenge, Newton et al. explored whether larger acoustic transmitters – also referred to as tags, impaired outmigration success. In order to do this, they implanted 68 salmon smolt (115 – 168mm FL) over two years with acoustic transmitters (1.9g mass) representing between 4 – 12.66% of body weight, and examined outmigration success in relation to fish length and mass to determine if transmitter size relative to fish size impaired outmigration success. Outmigration success was determined by fish passing through 2 acoustic receiver ‘gates’ – one at the end of the river outmigration (termed AT), and a second gate at the opening of the estuary, signifying the end of estuarine outmigration (termed ST).
Two-way Welch’s t-tests were performed on fish length, fish mass, and ratios of transmitter length to fish length, and transmitter mass to fish mass between 4 groups of fish: 1) those that succeeded in passing AT, 2), those that failed to pass AT, 3), those of group (1) that successfully passed ST, and finally 4), those of group (1) that failed to pass ST. The tests performed are: 1 vs 2 and 3 vs 4.
The study reported that there were no significant impacts of the transmitter on outmigration success, expressed as either differences in mean fish length or mass between successful and unsuccessful migrating individuals, or as a difference in mean relative transmitter length of transmitter mass (expressed as a proportion of the fish’s length or mass, respectively).
My main concern with this study relates to the statistical power of Newton et al.’s study, though I will also discuss the relative merits of the approach taken to explore the impacts of relative transmitter size on outmigration success.
Effect size and statistical power
I limit my analysis to fork length, but the findings here are relevant to the other metrics explored by the authors. Newton et al. reported that fork length was not significantly related to outmigration success, and thus concluded that transmitter size has little or no effect, at least over the size range and ecological context examined in their study. However, non-significant results could arise either because transmitters really do have minimal effect on survival, or because the study had insufficient power to detect an effect. One can compare these two interpretations by calculating effect size and its confidence limits. If effect size is close to zero and is bounded by tight confidence intervals, we can conclude that the study represents good evidence that larger transmitters do not compromise outmigration success. However, if effect size is bounded by large confidence intervals, which include zero as well as some large effect sizes, then we should conclude that the study is inconclusive: its statistical power was low enough that it is equally consistent with large effects and small (or no) effect.
Effect size
The authors do a fantastic job of presenting their results clearly: the treatment means, standard deviations and sample sizes are all presented in Table 1. Given that the authors present the means, standard deviations and sample sizes, we can calculate the effect size (Cohen’s d) and its 95% confidence interval using standard formulas, which are implemented in the mes function in the R package compute.es (Figure 1).
Given that outmigration by salmon smolts is a key life history challenge, Newton et al. explored whether larger acoustic transmitters – also referred to as tags, impaired outmigration success. In order to do this, they implanted 68 salmon smolt (115 – 168mm FL) over two years with acoustic transmitters (1.9g mass) representing between 4 – 12.66% of body weight, and examined outmigration success in relation to fish length and mass to determine if transmitter size relative to fish size impaired outmigration success. Outmigration success was determined by fish passing through 2 acoustic receiver ‘gates’ – one at the end of the river outmigration (termed AT), and a second gate at the opening of the estuary, signifying the end of estuarine outmigration (termed ST).
Two-way Welch’s t-tests were performed on fish length, fish mass, and ratios of transmitter length to fish length, and transmitter mass to fish mass between 4 groups of fish: 1) those that succeeded in passing AT, 2), those that failed to pass AT, 3), those of group (1) that successfully passed ST, and finally 4), those of group (1) that failed to pass ST. The tests performed are: 1 vs 2 and 3 vs 4.
The study reported that there were no significant impacts of the transmitter on outmigration success, expressed as either differences in mean fish length or mass between successful and unsuccessful migrating individuals, or as a difference in mean relative transmitter length of transmitter mass (expressed as a proportion of the fish’s length or mass, respectively).
My main concern with this study relates to the statistical power of Newton et al.’s study, though I will also discuss the relative merits of the approach taken to explore the impacts of relative transmitter size on outmigration success.
Effect size and statistical power
I limit my analysis to fork length, but the findings here are relevant to the other metrics explored by the authors. Newton et al. reported that fork length was not significantly related to outmigration success, and thus concluded that transmitter size has little or no effect, at least over the size range and ecological context examined in their study. However, non-significant results could arise either because transmitters really do have minimal effect on survival, or because the study had insufficient power to detect an effect. One can compare these two interpretations by calculating effect size and its confidence limits. If effect size is close to zero and is bounded by tight confidence intervals, we can conclude that the study represents good evidence that larger transmitters do not compromise outmigration success. However, if effect size is bounded by large confidence intervals, which include zero as well as some large effect sizes, then we should conclude that the study is inconclusive: its statistical power was low enough that it is equally consistent with large effects and small (or no) effect.
Effect size
The authors do a fantastic job of presenting their results clearly: the treatment means, standard deviations and sample sizes are all presented in Table 1. Given that the authors present the means, standard deviations and sample sizes, we can calculate the effect size (Cohen’s d) and its 95% confidence interval using standard formulas, which are implemented in the mes function in the R package compute.es (Figure 1).
Figure 1. Effect sizes for Fork length from Newton et al. (in press). These were computed using the mean, standard deviation and sample sizes reported in Table 1, and the function ‘mes’ in ‘compute.es’. The size of the square is proportional to the precision of the estimate.
We can see that the estimate of Cohen’s d implies there is little or no difference (d = 0.04) in fork length between successful and unsuccessful outmigrating salmon to AT. Successful migration to ST has a negative effect – Cohen’s d = -0.31 (meaning that successful fish were smaller than larger fish), however, what we are really interested in here, is the 95% confidence intervals.
However in both sites (AT and ST), the 95% confidence intervals in effects are wide. For example in AT, the 95% confidence interval is -0.46 – 0.53, meaning that the study cannot rule out the existence of a large negative effect, or a large positive effect, of transmitter size on the fork length of surviving fish.
This uncertainty is even greater at the ST site. Here, the sample sizes are smaller than at AT (n = 41), leading to a very wide 95% confidence limit: -1.11 – 0.49. This again tells us that the true effect could be highly positive, highly negative, or neutral – but we don’t know.
Statistical power
Ok, so if we were to repeat this study, what should we do to be sure we have a reasonable chance of detecting if there is an effect, or conversely, being confident that there isn’t one? Enter the power analysis.
Power analysis takes information regarding effect size, level of confidence and desired level of statistical power and gives an indication of the required sample size needed to meet these requirements. The R package ‘pwr’ can do all this.
Firstly, we can explore with what statistical power (1-β) the present study had to detect differences in mean length of successful and unsuccessful fish migrating at AT. Statistical power is defined as (1-β) (where β is the Type II error rate). Essentially, we want to have good power in order to reject the null hypothesis in the event that the null hypothesis is not supported. I choose to look at three levels of effect size: small (Cohen’s d = 0.3), medium (Cohen’s d = 0.5) and large (Cohen’s d = 0.8). It’s worth noting that Cohen’s d is defined as:
Cohen’s d = (μA – μB)/s
Where μA and μB are the sample means, and s is the pooled standard deviation.
Given our sample size and a Cohen’s d, we can compute the statistical power we would have with our current sample size of 68 fish (Table 1).
Table 1. Summary of statistical power to detect an effect at AT.
Cohen's d Power (1-β)
0.3 0.22
0.5 0.51
0.8 0.88
It should be noted that at ST (the final point of outmigration examined, the sample size had shrunk considerably (n = 41, and only 8 fish in the unsuccessful group). As such, statistical power at this point was even lower (Table 2).
Table 2. Summary of statistical power at outmigration point ST.
Cohen's d Power (1-β)
0.3 0.11
0.5 0.24
0.8 0.51
Ok, so what if we wanted to repeat this study, how many fish should we have tagged, to have a moderate degree of statistical power (1-β = 50%), to detect a small effect at AT (Cohen’s d = 0.3)?
Answer: n = 180 assuming an equal number of fish end up in the success and unsuccessful categories (ie. 50% mortality). If mortality varies greatly from 50%, a larger number of fish would be required.
What if we want a high degree of statistical power (80%) to be able to detect a small effect (Cohen’s d = 0.3) in fork length between successful and unsuccessful fish at AT?
Answer: 350 fish, assuming an equal number of fish in each category (50% mortality).
The question now becomes, what is a biologically important effect? Ultimately, this depends on the question that is being asked of the study, and that should dictate whether or not a small effect or a large effect is relevant. However, in this case, to be able to say that transmitters up to 12% of body size do not have an adverse impact on outmigration, a small effect is important in the context of this question. This is particularly valid if later studies are going to explore size-related impacts on outmigration success using transmitters. Even small transmitter-related effects may either artificially inflate or mask true yet small and biologically relevant fish length effect on outmigration success. Hence demonstrating that transmitter-related effects are negligible, with a high degree of certainty, is vital.
Other concerns
No “non-tagged” control
The study here makes the implicit assumption that there should be no difference in outmigration success with respect to fish size. Is this reasonable? A recurring theme in fisheries science is that ‘bigger is better’ and there is often ‘positive size-selective mortality’ – both of these imply that success in this instance should favour larger individuals. As such, we would naturally expect that smaller individuals would have a higher mortality rate. As such, is the null hypothesis (outmigration success is not a function of fish size) appropriate? Alternatively, examples exist where smaller fish are favoured (negative size-selective mortality) in which, is differential mortality being masked by natural processes?
The real issue here is that there is no true control group. We do not have any information from non-tagged fish to contextualise these results. For instance, is the observed mortality rate realistic? Should survival been a lot higher? Does tagging (irrespective of the relationship between tag size and fish size) cause increased mortality? If mortality is unusually high due to tagging, has this masked the true effect? Unfortunately, collecting such data is practically impossible!
Non-measured traits
The present study looked at 2 traits – fish length, and fish mass. But are these really the most appropriate metrics to explore? Over the range of transmitter sizes examined, perhaps fish condition (rather than length or mass independently) would have been a more relevant metric to examine. Again, a suitable control (eg, non-tagged fish) would be required to fully interpret the results. Outmigration success may be influenced by swimming performance, for which there are a range of possible swimming related traits that could be examined. Growth and survival is a whole new ‘kettle of fish’ that hasn’t been considered, but is certainly ecologically relevant.
Longevity of study
The present study looked at outmigration over a period of 44 days. In the context of studying outmigration and if we are not interested in their fate thereafter, this is fine. However, extrapolating these findings to other studies would be risky. The longer term impacts of carrying a transmitter fairly were not the focus of this study, but warrants examination.
Acknowledgements: Thanks to Dr Luke Holman (University of Melbourne) for helpful discussion, comments and pointing out the R package ‘pwr’.
1 Newton, M., Barry, J., Dodd, J.A., Lucas, M.C., Boylan, P., and C.E. Adams (in press) Does size matter? A test of size-specific mortality in Atlantic salmon Salmo salar smolts tagged with acoustic transmitters. Journal of Fish Biology doi: 10.1111/jfb.13066
R code (Windows, RStudio)
##Load packages
library(compute.es)
library(pwr)
library(metafor) ##for forest plot
##Compute effect sizes from study##
AT_ES <- mes(138.8, 138.3, 12.7, 13.8, 41, 27)
ST_ES <- mes(139.1, 143.0, 12.2, 13.5, 33, 8)
##Forest plot
##extract Cohen's d and variance of cohen's d from effect size analysis (AT_ES and ST_ES)
Cohen_D <- cbind(0.04, -0.31)
Cohen_Var <- cbind(0.06, 0.16)
D <- as.vector(Cohen_D)
Var <- as.vector(Cohen_Var)
forest(D, Var)
###Power analysis
## Outmigration point AT
P1_AT <- pwr.t2n.test(n1 = 41, n2 = 27, d = 0.3, sig.level = 0.05, power = NULL, alternative = c("two.sided"))
plot.power.htest(P1_AT)
P2_AT <- pwr.t2n.test(n1 = 41, n2 = 27, d = 0.5, sig.level = 0.05, power = NULL, alternative = c("two.sided"))
plot.power.htest(P2_AT)
P3_AT <- pwr.t2n.test(n1 = 41, n2 = 27, d = 0.8, sig.level = 0.05, power = NULL, alternative = c("two.sided"))
plot.power.htest(P3_AT)
## Outmigration point ST
P1_ST <- pwr.t2n.test(n1 = 33, n2 = 8, d = 0.3, sig.level = 0.05, power = NULL, alternative = c("two.sided"))
plot.power.htest(P1_ST)
P2_ST <- pwr.t2n.test(n1 = 33, n2 = 8, d = 0.5, sig.level = 0.05, power = NULL, alternative = c("two.sided"))
plot.power.htest(P2_ST)
P3_ST <- pwr.t2n.test(n1 = 33, n2 = 8, d = 0.8, sig.level = 0.05, power = NULL, alternative = c("two.sided"))
plot.power.htest(P3_ST)
##Estimating new sample sizes
Medium_AT <- pwr.t.test(n=NULL, d=0.3, sig.level = 0.05, power = 0.5, alternative = c("two.sided"))
plot.power.htest(Medium_AT)
High_AT <- pwr.t.test(n=NULL, d=0.3, sig.level = 0.05, power = 0.8, alternative = c("two.sided"))
plot.power.htest(High_AT)
However in both sites (AT and ST), the 95% confidence intervals in effects are wide. For example in AT, the 95% confidence interval is -0.46 – 0.53, meaning that the study cannot rule out the existence of a large negative effect, or a large positive effect, of transmitter size on the fork length of surviving fish.
This uncertainty is even greater at the ST site. Here, the sample sizes are smaller than at AT (n = 41), leading to a very wide 95% confidence limit: -1.11 – 0.49. This again tells us that the true effect could be highly positive, highly negative, or neutral – but we don’t know.
Statistical power
Ok, so if we were to repeat this study, what should we do to be sure we have a reasonable chance of detecting if there is an effect, or conversely, being confident that there isn’t one? Enter the power analysis.
Power analysis takes information regarding effect size, level of confidence and desired level of statistical power and gives an indication of the required sample size needed to meet these requirements. The R package ‘pwr’ can do all this.
Firstly, we can explore with what statistical power (1-β) the present study had to detect differences in mean length of successful and unsuccessful fish migrating at AT. Statistical power is defined as (1-β) (where β is the Type II error rate). Essentially, we want to have good power in order to reject the null hypothesis in the event that the null hypothesis is not supported. I choose to look at three levels of effect size: small (Cohen’s d = 0.3), medium (Cohen’s d = 0.5) and large (Cohen’s d = 0.8). It’s worth noting that Cohen’s d is defined as:
Cohen’s d = (μA – μB)/s
Where μA and μB are the sample means, and s is the pooled standard deviation.
Given our sample size and a Cohen’s d, we can compute the statistical power we would have with our current sample size of 68 fish (Table 1).
Table 1. Summary of statistical power to detect an effect at AT.
Cohen's d Power (1-β)
0.3 0.22
0.5 0.51
0.8 0.88
It should be noted that at ST (the final point of outmigration examined, the sample size had shrunk considerably (n = 41, and only 8 fish in the unsuccessful group). As such, statistical power at this point was even lower (Table 2).
Table 2. Summary of statistical power at outmigration point ST.
Cohen's d Power (1-β)
0.3 0.11
0.5 0.24
0.8 0.51
Ok, so what if we wanted to repeat this study, how many fish should we have tagged, to have a moderate degree of statistical power (1-β = 50%), to detect a small effect at AT (Cohen’s d = 0.3)?
Answer: n = 180 assuming an equal number of fish end up in the success and unsuccessful categories (ie. 50% mortality). If mortality varies greatly from 50%, a larger number of fish would be required.
What if we want a high degree of statistical power (80%) to be able to detect a small effect (Cohen’s d = 0.3) in fork length between successful and unsuccessful fish at AT?
Answer: 350 fish, assuming an equal number of fish in each category (50% mortality).
The question now becomes, what is a biologically important effect? Ultimately, this depends on the question that is being asked of the study, and that should dictate whether or not a small effect or a large effect is relevant. However, in this case, to be able to say that transmitters up to 12% of body size do not have an adverse impact on outmigration, a small effect is important in the context of this question. This is particularly valid if later studies are going to explore size-related impacts on outmigration success using transmitters. Even small transmitter-related effects may either artificially inflate or mask true yet small and biologically relevant fish length effect on outmigration success. Hence demonstrating that transmitter-related effects are negligible, with a high degree of certainty, is vital.
Other concerns
No “non-tagged” control
The study here makes the implicit assumption that there should be no difference in outmigration success with respect to fish size. Is this reasonable? A recurring theme in fisheries science is that ‘bigger is better’ and there is often ‘positive size-selective mortality’ – both of these imply that success in this instance should favour larger individuals. As such, we would naturally expect that smaller individuals would have a higher mortality rate. As such, is the null hypothesis (outmigration success is not a function of fish size) appropriate? Alternatively, examples exist where smaller fish are favoured (negative size-selective mortality) in which, is differential mortality being masked by natural processes?
The real issue here is that there is no true control group. We do not have any information from non-tagged fish to contextualise these results. For instance, is the observed mortality rate realistic? Should survival been a lot higher? Does tagging (irrespective of the relationship between tag size and fish size) cause increased mortality? If mortality is unusually high due to tagging, has this masked the true effect? Unfortunately, collecting such data is practically impossible!
Non-measured traits
The present study looked at 2 traits – fish length, and fish mass. But are these really the most appropriate metrics to explore? Over the range of transmitter sizes examined, perhaps fish condition (rather than length or mass independently) would have been a more relevant metric to examine. Again, a suitable control (eg, non-tagged fish) would be required to fully interpret the results. Outmigration success may be influenced by swimming performance, for which there are a range of possible swimming related traits that could be examined. Growth and survival is a whole new ‘kettle of fish’ that hasn’t been considered, but is certainly ecologically relevant.
Longevity of study
The present study looked at outmigration over a period of 44 days. In the context of studying outmigration and if we are not interested in their fate thereafter, this is fine. However, extrapolating these findings to other studies would be risky. The longer term impacts of carrying a transmitter fairly were not the focus of this study, but warrants examination.
Acknowledgements: Thanks to Dr Luke Holman (University of Melbourne) for helpful discussion, comments and pointing out the R package ‘pwr’.
1 Newton, M., Barry, J., Dodd, J.A., Lucas, M.C., Boylan, P., and C.E. Adams (in press) Does size matter? A test of size-specific mortality in Atlantic salmon Salmo salar smolts tagged with acoustic transmitters. Journal of Fish Biology doi: 10.1111/jfb.13066
R code (Windows, RStudio)
##Load packages
library(compute.es)
library(pwr)
library(metafor) ##for forest plot
##Compute effect sizes from study##
AT_ES <- mes(138.8, 138.3, 12.7, 13.8, 41, 27)
ST_ES <- mes(139.1, 143.0, 12.2, 13.5, 33, 8)
##Forest plot
##extract Cohen's d and variance of cohen's d from effect size analysis (AT_ES and ST_ES)
Cohen_D <- cbind(0.04, -0.31)
Cohen_Var <- cbind(0.06, 0.16)
D <- as.vector(Cohen_D)
Var <- as.vector(Cohen_Var)
forest(D, Var)
###Power analysis
## Outmigration point AT
P1_AT <- pwr.t2n.test(n1 = 41, n2 = 27, d = 0.3, sig.level = 0.05, power = NULL, alternative = c("two.sided"))
plot.power.htest(P1_AT)
P2_AT <- pwr.t2n.test(n1 = 41, n2 = 27, d = 0.5, sig.level = 0.05, power = NULL, alternative = c("two.sided"))
plot.power.htest(P2_AT)
P3_AT <- pwr.t2n.test(n1 = 41, n2 = 27, d = 0.8, sig.level = 0.05, power = NULL, alternative = c("two.sided"))
plot.power.htest(P3_AT)
## Outmigration point ST
P1_ST <- pwr.t2n.test(n1 = 33, n2 = 8, d = 0.3, sig.level = 0.05, power = NULL, alternative = c("two.sided"))
plot.power.htest(P1_ST)
P2_ST <- pwr.t2n.test(n1 = 33, n2 = 8, d = 0.5, sig.level = 0.05, power = NULL, alternative = c("two.sided"))
plot.power.htest(P2_ST)
P3_ST <- pwr.t2n.test(n1 = 33, n2 = 8, d = 0.8, sig.level = 0.05, power = NULL, alternative = c("two.sided"))
plot.power.htest(P3_ST)
##Estimating new sample sizes
Medium_AT <- pwr.t.test(n=NULL, d=0.3, sig.level = 0.05, power = 0.5, alternative = c("two.sided"))
plot.power.htest(Medium_AT)
High_AT <- pwr.t.test(n=NULL, d=0.3, sig.level = 0.05, power = 0.8, alternative = c("two.sided"))
plot.power.htest(High_AT)
RSS Feed
