Have a personal or library account? Click to login
Do You See What I See? Quantifying Inter-Observer Variability in an Intertidal Marine Citizen Science Experiment Cover

Do You See What I See? Quantifying Inter-Observer Variability in an Intertidal Marine Citizen Science Experiment

Citizen Science: Theory and Practice
Volume 7 (2022): Issue 1
By: Hannah Earp,  Siobhan Vye,  Katrin Bohn,  Michael Burrows,  Jade Chenery,  Stephanie Dickens,  Charlotte Foster,  Hannah Grist,  Peter Lamont,  Sarah Long,  Zoe Morrall,  Jacqueline Pocklington,  Abigail Scott,  Gordon Watson,  Victoria West,  Stuart Jenkins,  Jane Delany and  Heather Sugden  
Open Access
|May 2022

References

  1. 1Balázs, B, Mooney, P, Nováková, E, Bastin, L and Arsanjani, JJ. 2021. The Science of Citizen Science. In: Vohland, K, Land-Zandstra, A, Ceccaroni, L, Lemmens, R, Perello, J, Ponti, M, Samon, R, and Wagenknecht, K (eds.) The Science of Citizen Science. Cham: Springer. Pp. 139–197. DOI: 10.1145/3022198.3022652
    Back to article
  2. 2Bates, D, Mächler, M, Bolker, B and Walker, S. 2015. Fitting linear mixed-effects models using lme4, Journal of Statistical Software, 67(1): 1–48. DOI: 10.18637/jss.v067.i01
    Back to article
  3. 3Bonney, R, Cooper, CB, Dickinson, J, Kelling, S, Phillips, T, Rosenberg, KV and Shirk, J. 2009. Citizen science: A developing tool for expanding science knowledge and scientific literacy. BioScience, 59(11): 977–984. DOI: 10.1525/bio.2009.59.11.9
    Back to article
  4. 4Bonney, R, Shirk, JL, Phillips, TB, Wiggins, A, Ballard, HL, Miller-Rushing, AJ and Parrish, JK. 2014. Citizen science: Next steps for citizen science. Science, 343(6178): 1436–1437. DOI: 10.1126/science.1251554
    Back to article
  5. 5Branchini, S, Pensa, F, Neri, P., Tonucci, BM, Mattielli, L, Collavo, A, Sillingardi, ME, Piccinetti, C, Zaccanti, F and Goffredo, S. 2015. Using a citizen science program to monitor coral reef biodiversity through space and time. Biodiversity and Conservation, 24(2): 319–336. DOI: 10.1007/s10531-014-0810-7
    Back to article
  6. 6Burgess, HK, Debey, LB, Froehlich, HE, Schmidt, N, Theobald, EJ, Ettinger, AK, Hillerislambers, J, Tewksbury, J and Parrish, JK. 2017. The science of citizen science: Exploring barriers to use as a primary research tool. Biological Conservation, 208: 113–120. DOI: 10.1016/j.biocon.2016.05.014
    Back to article
  7. 7Burrows, MT. 2012. Influences of wave fetch, tidal flow and ocean colour on subtidal rocky communities. Marine Ecology Progress Series, 445: 193-U462. DOI: 10.3354/meps09422
    Back to article
  8. 8Burrows, MT, 2020. Wave fetch GIS layers for Europe at 100m scale, Figshare. Dataset, DOI: 10.6084/m9.figshare.8668127.
    Back to article
  9. 9Carleton, JH and Done, TJ. 1995. Quantitative video sampling of coral reef benthos: large-scale application. Coral Reefs, 14: 35–46. DOI: 10.1007/BF00304070
    Back to article
  10. 10Cerrano, C, Milanese, M and Ponti, M. 2017. Diving for science – science for diving: volunteer scuba divers support science and conservation in the Mediterranean Sea. Aquatic Conservation: Marine and Freshwater Ecosystems. DOI: 10.1002/aqc.2663
    Back to article
  11. 11Clare, JDJ, Townsend, PA, Anhalt-Depies, C, Locke, C, Stenglein, JL, Frett, S, Martin, KJ, Singh, A, Van Deelen, TR and Zuckerberg, B. 2019. Making inference with messy (citizen science) data: when are data accurate enough and how can they be improved?. Ecological Applications, e01849. DOI: 10.1002/eap.1849
    Back to article
  12. 12Cox, TE, Philippoff, J, Baumgartner, E and Smith, CM. 2012. Expert variability provides perspective on the strengths and weaknesses of citizen-driven intertidal monitoring program. Ecological Applications, 22(4): 1201–1212. DOI: 10.1890/11-1614.1
    Back to article
  13. 13Crall, AW, Newman, GJ, Stohlgren, TJ, Holfelder, KA, Graham, J and Waller, DM. 2011. Assessing citizen science data quality: An invasive species case study. Conservation Letters, 4(6): 433–442. DOI: 10.1111/j.1755-263X.2011.00196.x
    Back to article
  14. 14Delaney, DG, Sperling, CD, Adams, CS and Leung, B. 2008. Marine invasive species: Validation of citizen science and implications for national monitoring networks. Biological Invasions, 10(1): 117–128. DOI: 10.1007/s10530-007-9114-0
    Back to article
  15. 15Dethier, MN, Graham, ES, Cohen, S and Tear, LM. 1993. Visual versus random-point percent cover estimations: ‘objective’ is not always better. Marine Ecology Progress Series, 96(1): 93–100. DOI: 10.3354/meps096093
    Back to article
  16. 16Dickinson, JL, Zuckerberg, B and Bonter, DN. 2010. Citizen science as an ecological research tool: challenges and benefits. Annual Review of Ecology, Evolution, and Systematics, 41(1): 149–172. DOI: 10.1146/annurev-ecolsys-102209-144636
    Back to article
  17. 17Durden, JM, Bett, BJ, Schoening, T, Morris, KJ, Nattkemper, TW and Ruhl, HA. 2016. Comparison of image annotation data generated by multiple investigators for benthic ecology. Marine Ecology Progress Series, 552: 61–70. DOI: 10.3354/meps11775
    Back to article
  18. 18Earp, HS and Liconti, A. 2020. Science for the future: The use of citizen science in marine research and conservation, In: Jungblut, S, Leibich, V, and Bode-Dalby, M (eds.). YOUMARES 9 – The cceans: Our research, our future. Cham: Springer. Pp 1–19. DOI: 10.1007/978-3-030-20389-4_1
    Back to article
  19. 19Edgar, GJ and Stuart-Smith, RD. 2009. Ecological effects of marine protected areas on rocky reef communities – A continental-scale analysis. Marine Ecology Progress Series, 388(4): 51–62. DOI: 10.3354/meps08149
    Back to article
  20. 20Evans, SM, Gebbels, S and Stockill, JM. 2008. ‘Our shared responsibility’: Participation in ecological projects as a means of empowering communities to contribute to coastal management processes. Marine Pollution Bulletin, 57(1–5): 3–7. DOI: 10.1016/j.marpolbul.2008.04.014
    Back to article
  21. 21Finn, PG, Udy, NS, Baltais, SJ, Price, K and Coles, L. 2010. Assessing the quality of seagrass data collected by community volunteers in Moreton Bay Marine Park, Australia. Environmental Conservation, 37(1): 83–89. DOI: 10.1017/S0376892910000251
    Back to article
  22. 22Fore, LS, Paulsen, K and O’Laughlin, K. 2001. Assessing the performance of volunteers in monitoring streams. Freshwater Biology, 46(1): 109–123. DOI: 10.1046/j.1365-2427.2001.00640.x
    Back to article
  23. 23Forrester, G, Baily, P, Conetta, D, Forrester, L, Kintzing, E and Jarecki, L. 2015. Comparing monitoring data collected by volunteers and professionals shows that citizen scientists can detect long-term change on coral reefs. Journal for Nature Conservation, 24(C): 1–9. DOI: 10.1016/j.jnc.2015.01.002
    Back to article
  24. 24Fox, J and Weisberg, S, 2019. An R Companion to Applied Regression (Third Edit). Thousand Oaks, CA: Sage.
    Back to article
  25. 25Freitag, A, Meyer, R and Whiteman, L. 2016. Strategies employed by citizen science programs to increase the credibility of their data. Citizen Science: Theory and Practice, 1(1). DOI: 10.5334/cstp.6
    Back to article
  26. 26Galloway, AWE, Tudor, MT and Vander Haegen, WM. 2006. The reliability of citizen science: A case study of oregon white oak stand surveys. Wildlife Society Bulletin, 34(5): 1425–1429. DOI: 10.2193/0091-7648(2006)34[1425:TROCSA]2.0.CO;2
    Back to article
  27. 27Garcia-Soto, C, Seys, JJC., Zielinski, O, Busch, JA, Luna, SI, Baez, JC, Domegan, C, Dubsky, K, Kotynska-Zielinska, I, Loubat, P, Malfatti, F, Mannaerts, G, McHugh, P, Monestiez, P, van der Meeren, GI and Gorsky, G. 2021. Marine citizen science: Current state in Europe and new technological developments. Frontiers in Marine Science, 8: 1–13. DOI: 10.3389/fmars.2021.621472
    Back to article
  28. 28Gardiner, M.M, Allee, LL, Brown, PMJ, Losey, JE, Roy, HE and Smyth, RR. 2012. Lessons from lady beetles: Accuracy of monitoring data from US and UK citizen science programs. Frontiers in Ecology and the Environment, 10(9): 471–476. DOI: 10.1890/110185
    Back to article
  29. 29Gardner, H. 1993. Multiple intelligences: The theory in practice. New York: Basic Books.
    Back to article
  30. 30Gillett, DJ, Pondella, DJ, Freiwald, J, Schiff, KC, Caselle, JE, Shuman, C and Weisberg, SB. 2012. Comparing volunteer and professionally collected monitoring data from the rocky subtidal reefs of southern California, USA. Environmental Monitoring and Assessment, 184(5): 3239–3257. DOI: 10.1007/s10661-011-2185-5
    Back to article
  31. 31Gollan, J, De Bruyn, LL, Reid, N and Wilkie, L. 2012. Can volunteers collect data that are comparable to professional scientists? a study of variables used in monitoring the outcomes of ecosystem rehabilitation. Environmental Management, 50(5): 969–978. DOI: 10.1007/s00267-012-9924-4
    Back to article
  32. 32Halpern, BS, Walbridge, S, Selkoe, KA, Kappel, CV, Micheli, F, D’Agrosa, C, Bruno, JF, Casey, KS, Ebert, C, Fox, HE, Fujita, R, Heinemann, D, Lenihan, HS, Madin, EMP, Perry, MT, Selig, ER., Spalding, M, Steneck, R and Watson, R. 2008. A global map of human impact on marine ecosystems. Science, 319(5865): 948–952. DOI: 10.1126/science.1149345
    Back to article
  33. 33Hidalgo-Ruz, V and Thiel, M. 2013. Distribution and abundance of small plastic debris on beaches in the SE Pacific (Chile): A study supported by a citizen science project. Marine Environmental Research, 87–88: 12–18. DOI: 10.1016/j.marenvres.2013.02.015
    Back to article
  34. 34Hothon, T, Bretz, F and Westfall, P. 2008. Simultaneous inference in general parametric models. Biometrical Journal, 50(3): 346–363. DOI: 10.1002/bimj.200810425
    Back to article
  35. 35Kohler, KE and Gill, SM. 2006. Coral Point Count with Excel extensions (CPCe): A Visual Basic program for the determination of coral and substrate coverage using random point count methodology. Computers and Geosciences, 32(9): 1259–1269. DOI: 10.1016/j.cageo.2005.11.009
    Back to article
  36. 36Kosmala, M, Wiggins, A, Swanson, A and Simmons, B. 2016. Assessing data quality in citizen science. Frontiers in Ecology and the Environment, 14(10): 551–560. DOI: 10.1002/fee.1436
    Back to article
  37. 37Koss, RS, Miller, K, Wescott, G, Bellgrove, A, Boxshall, A, McBurnie, J, Bunce, A, Gilmour, P and Ierodiaconou, D. 2009. An evaluation of Sea Search as a citizen science programme in marine protected areas. Pacific Conservation Biology, 15(2): 116–127. DOI: 10.1071/PC090116
    Back to article
  38. 38Magurran, AE, Baillie, SR, Buckland, ST, Dick, JMP, Elston, DA, Scott, EM, Smith, RI, Somerfield, PJ and Watt, AD. 2010. Long-term datasets in biodiversity research and monitoring: Assessing change in ecological communities through time. Trends in Ecology and Evolution, 25(10): 574–582. DOI: 10.1016/j.tree.2010.06.016
    Back to article
  39. 39Marshall, NJ, Kleine, DA and Dean, AJ. 2012. CoralWatch: Education, monitoring, and sustainability through citizen science. Frontiers in Ecology and the Environment, 10(6): 332–334. DOI: 10.1890/110266
    Back to article
  40. 40Meese, RJ and Tomich, PA. 1992. Dots on the rocks: a comparison of percent cover estimation methods. Journal of Experimental Marine Biology and Ecology, 165(1): 59–73. DOI: 10.1016/0022-0981(92)90289-M
    Back to article
  41. 41Mieszkowska, N, Sugden, H, Firth, LB and Hawkins, SJ. 2014. The role of sustained observations in tracking impacts of environmental change on marine biodiversity and ecosystems. Philosophical Transactions of the Royal Society A, 372(20130339). DOI: 10.1098/rsta.2013.0339
    Back to article
  42. 42Mumby, PJ, Harborne, AR, Raines, PS and Ridley, JM. 1995. A critical assessment of data derived from coral cay conservation volunteers. Bulletin of Marine Science, 56(3): 737–751
    Back to article
  43. 43O’Brien, L, Townsend, M and Ebden, M. 2010. ‘Doing something positive’: volunteers’ experiences of the well-being benefits derived from practical conservation activities in nature. Voluntas, 21(4): 525–545. DOI: 10.1007/s11266-010-9149-1
    Back to article
  44. 44Parsons, J, Lukyanenko, R and Wiersma, Y. 2011. Easier citizen science is better. Nature, 471(7336): 37–37. DOI: 10.1038/471037a
    Back to article
  45. 45Pecl, GT, Araújo, MB, Bell, JD, Blanchard, J, Bonebrake, TC, Chen, IC, Clark, TD, Colwell, RK, Danielsen, F, Evengård, B, Falconi, L, Ferrier, S, Frusher, S, Garcia, RA, Griffis, RB, Hobday, AJ, Janion-Scheepers, C, Jarzyna, MA, Jennings, S, Lenoir, J, Linnetved, HI, Martin, VY, McCormack, PC, McDonald, J, Mitchell, NJ, Mustonen, T, Pandolfi, JM, Pettorelli, N, Popova, E, Robinson, SA, Scheffers, BR, Shaw, JD, Sorte, CJB, Strugnell, JM, Sunday, JM, Tuanmu, MN, Vergés, A, Villanueva, C, Wernberg, T, Wapstra, E and Williams, SE. 2017. Biodiversity redistribution under climate change: Impacts on ecosystems and human well-being. Science, 355(6332). DOI: 10.1126/science.aai9214
    Back to article
  46. 46Penrose, D and Call, SM. 1995. Volunteer monitoring of benthic macroinvertebrates: regulatory biologists’ perspectives. Journal of the North American Benthological Society, 14(1): 203–209. DOI: 10.2307/1467735
    Back to article
  47. 47Poloczanska, ES, Brown, CJ, Sydeman, WJ, Kiessling, W, Schoeman, DS, Moore, PJ, Brander, K, Bruno, JF, Buckley, LB, Burrows, MT, Duarte, CM, Halpern, BS, Holding, J, Kappel, CV, O’Connor, MI, Pandolfi, JM, Parmesan, C, Schwing, F, Thompson, SA and Richardson, AJ. 2013. Global imprint of climate change on marine life. Nature Climate Change, 3(10): 919–925. DOI: 10.1038/nclimate1958
    Back to article
  48. 48R Core Team. 2021. R: A language and environment for statistical computing. R Foundation for Statistical Computing.
    Back to article
  49. 49Rambonnet, L, Vink, SC, Land-Zandstra, AM and Bosker, T. 2019. Making citizen science count: Best practices and challenges of citizen science projects on plastics in aquatic environments. Marine Pollution Bulletin, 145: 271–277. DOI: 10.1016/j.marpolbul.2019.05.056
    Back to article
  50. 50Rosenthal, IS, Byrnes, JE, Cavanaugh, KC, Bell, TW, Harder, B, Haupt, AJ, Rassweiler, ATW, Perez-Matus, A, Assis, J, Swanson, A, Boyer, A, McMaster, A and Trouille, L. 2018. Floating forests: Quantitative validation of citizen science data generated from consensus classifications. ArXiv Preprint, 1801.08522.
    Back to article
  51. 51Sandahl, A and Tøttrup, AP. 2020. Marine citizen science: Recent developments and future recommendations. Citizen Science: Theory and Practice, 5(1): 1–11. DOI: 10.5334/CSTP.270
    Back to article
  52. 52Simpson, R, Page, KR and De Roure, D. 2014. Zooniverse: Observing the world’s largest citizen science platform. In: Proceedings of the 23rd International Conference on World Wide Web, 2014, pp. 1049–1054. DOI: 10.1145/2567948.2579215
    Back to article
  53. 53Swanson, A, Kosmala, M, Lintott, C and Packer, C. 2016. A generalized approach for producing, quantifying, and validating citizen science data from wildlife images. Conservation Biology, 30(3): 520–531. DOI: 10.1111/cobi.12695
    Back to article
  54. 54Thiel, M, Penna-Díaz, M, Luna-Jorquera, G, Salas, S, Sellanes, J and Stotz, W. 2014. Citizen scientists and marine research: volunteer participants, their contributions, and projection for the future. Oceanography and Marine Biology: An Annual Review, 52: 257–314. DOI: 10.1201/b17143-6
    Back to article
  55. 55van der Velde, T, Milton, DA, Lawson, TJ, Wilcox, C, Lansdell, M, Davis, G, Perkins, G and Hardesty, BD. 2017. Comparison of marine debris data collected by researchers and citizen scientists: Is citizen science data worth the effort?. Biological Conservation, 208: 127–138. DOI: 10.1016/j.biocon.2016.05.025
    Back to article
  56. 56Vye, SR, Dickens, S, Adams, L, Bohn, K, Chenery, J, Dobson, N, Dunn, RE, Earp, HS, Evans, M, Foster, C, Grist, H, Holt, B, Hull, S, Jenkins, SR, Lamont, P, Long, S, Mieszkowska, N, Millard, J, Morrall, Z, Pack, K, Parry-Wilson, H, Pocklington, J, Pottas, J, Richardson, L, Scott, A, Sugden, H, Watson, G, West, V, Winton, D, Delany, J and Burrows, MT. 2020. Patterns of abundance across geographical ranges as a predictor for responses to climate change: Evidence from UK rocky shores. Diversity and Distributions, 26(10): 1357–1365. DOI: 10.1111/ddi.13118
    Back to article
  57. 57Warton, DI and Hui, FKC. 2011. The arcsine is asinine: The analysis of proportions in ecology. Ecology, 92(1): 3–10. DOI: 10.1890/10-0340.1
    Back to article
  58. 58Wickham, H. 2016. Elegant graphics for gata analysis. New York: Springer-Verlag.
    Back to article
  59. 59Zettler, ER, Takada, H, Monteleone, B, Mallos, N, Eriksen, M and Amaral-Zettler, LA. 2017. Incorporating citizen science to study plastics in the environment. Analytical Methods, 9(9): 1392–1403. DOI: 10.1039/C6AY02716D
    Back to article
DOI: https://doi.org/10.5334/cstp.483 | Journal eISSN: 2057-4991
Journal RSS Feed
Language: English
Submitted on: Dec 14, 2021
Accepted on: Apr 1, 2022
Published on: May 4, 2022
Published by: Ubiquity Press
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year
Keywords:
data verification,
data accuracy,
public participation,
volunteer,
Coral Point Count,
temperate rocky shore

© 2022 Hannah Earp, Siobhan Vye, Katrin Bohn, Michael Burrows, Jade Chenery, Stephanie Dickens, Charlotte Foster, Hannah Grist, Peter Lamont, Sarah Long, Zoe Morrall, Jacqueline Pocklington, Abigail Scott, Gordon Watson, Victoria West, Stuart Jenkins, Jane Delany, Heather Sugden, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.

Previous articleVolume 7 (2022): Issue 1Next article