From mailbox to inbox: mail vs internet waterfowl hunter survey responses
Jerry J. Vaske
A * , Eric M. Walberg B , Brent D. Williams B , Samantha G. Pallazza B , Lauren J. Stephens B and Craig A. Miller B
A
B
Abstract
As response rates from traditional mail surveys have declined, and the costs of conducting such surveys have increased, some social scientists have transitioned to internet surveys as a cost-effective alternative to mail surveys.
This study compared responses from mail and internet surveys. We predicted that the two surveys will differ in terms of response rates (Hypothesis 1), and item nonresponse (Hypothesis 2). Response distribution differences for the two samples were examined for two demographic variables: availability of internet access (Hypothesis 3) and age (Hypothesis 4). Avidity related variables included years of hunting waterfowl in Illinois (Hypothesis 5), days of waterfowl hunting (Hypothesis 6), ducks or geese harvested (Hypothesis 7) and satisfaction with the most recent waterfowl hunting experience in Illinois (Hypothesis 8).
We compared mail (n = 1,602) and internet (n = 816) surveys from waterfowl hunters in Illinois. The study examined three types of dependent variables: (1) response rates, (2) item nonresponse, and (3) questionnaire response distributions for demographics, avidity indicators, and satisfaction with waterfowl hunting experience in Illinois.
Of the eight hypotheses, only four were statistically significant. Statistical differences were observed for item nonresponse, internet access, age, and years of hunting waterfowl in Illinois. No statistical differences were found for response rates, days hunting, waterfowl harvest during the 2022–2023 season, and satisfaction with the most recent experience.
Results indicated that internet surveys can be cost-effective and a potential alternative to mail surveys in certain contexts.
Reductions in response rates and increasing costs of traditional mail surveys have led state wildlife agencies to explore alternative approaches to data collection, such as internet surveys. Although 50% (4 of 8) of the hypothesized mail versus internet survey comparisons differed statistically, the effect sizes were generally minimal. Ensuring that the survey mode effectively captures a representative sample of hunters is vital for future data collection efforts.
Keywords: avidity, demographics, hunter satisfaction, internet surveys, mail surveys, survey quality indicators, waterfowl hunters, waterfowl hunting experiences.
References
Aebischer NJ (2019) Fifty-year trends in UK hunting bags of birds and mammals, and calibrated estimation of national bag size, using GWCT’s National Gamebag Census. European Journal of Wildlife Research 65, 64.
| Crossref | Google Scholar |
Alessi MG, Miller CA (2012) Internet use by Illinois hunters: ten years after. Human Dimensions of Wildlife 17, 311-312.
| Crossref | Google Scholar |
Anderson L, Jans M, Lee A, Doyle C, Driscoll H, Hilger J (2022) Effects of survey response mode, purported topic, and incentives on response rates in human dimensions of fisheries and wildlife research. Human Dimensions of Wildlife 27(3), 201-219.
| Crossref | Google Scholar |
Antoun C, Couper MP, Conrad FG (2017) Effects of mobile versus PC web on survey response quality: a crossover experiment in a probability web panel. Public Opinion Quarterly 81(S1), 280-306.
| Crossref | Google Scholar |
Archer TM (2003) Web-based surveys. Journal of Extension 41(1), Article 25.
| Google Scholar |
Aubry P, Guillemain M (2019) Attenuating the nonresponse bias in hunting bag surveys: the multiphase sampling strategy. PLoS ONE 14(3), e0213670.
| Crossref | Google Scholar |
Barrios M, Villarroya A, Borrego Á, Ollé C (2011) Response rates and data quality in web and mail surveys administered to PhD holders. Social Science Computer Review 29(2), 208-220.
| Crossref | Google Scholar |
Bryan H (1977) Leisure value systems and recreational specialization: the case of trout fishermen. Journal of Leisure Research 9, 174-187.
| Crossref | Google Scholar |
Bybee S, Cloyes K, Ellington L, Baucom B, Supiano K, Mooney K (2022) Bots and nots: safeguarding online survey research with underrepresented and diverse populations. Psychology & Sexuality 13(4), 901-911.
| Crossref | Google Scholar | PubMed |
Cole ST (2005) Comparing mail and web-based survey distribution methods: results of surveys to leisure travel retailers. Journal of Travel Research 43, 422-430.
| Crossref | Google Scholar |
Converse PD, Wolfe EW, Huang X, Oswald FL (2008) Response rates for mixed-mode surveys using mail and E-mail/Web. American Journal of Evaluation 29, 99-107.
| Crossref | Google Scholar |
Cornicelli L, Grund MD (2011) Assessing deer hunter attitudes towards regulatory change using self-selected respondents. Human Dimensions of Wildlife 16, 174-182.
| Crossref | Google Scholar |
Couper MP, Blair J, Triplett T (1999) A comparison of mail and E-mail for a survey of employees in federal statistical agencies. Journal of Official Statistics 15, 39-56.
| Google Scholar |
Daikeler J, Bošnjak M, Lozar Manfreda K (2020) Web versus other survey modes: an updated and extended meta-analysis comparing response rates. Journal of Survey Statistics and Methodology 8(3), 513-539.
| Crossref | Google Scholar |
de Leeuw ED, Hox J, Huisman M (2003) Prevention and treatment of item nonresponse. Journal of Official Statistics 19, 153-176.
| Google Scholar |
Diment K, Garrett-Jones S (2007) How demographic characteristics affect mode preference in a Postal/Web mixed-mode survey of Australian researchers. Social Science Computer Review 25, 410-417.
| Crossref | Google Scholar |
Duda MD, Nobile JL (2010) The fallacy of online surveys: no data are better than bad data. Human Dimensions of Wildlife 15, 55-64.
| Crossref | Google Scholar |
Gamborg C, Lund JF, Jensen FS (2019) Landowners’ wildlife value orientations, attitudes and behaviour in relation to game management practices. European Journal of Wildlife Research 65, 9.
| Crossref | Google Scholar |
Gaynor KM, McInturff A, Abrahms BL, Smith AM, Brashares JS (2024) Hunting mode and habitat selection mediate the success of human hunters. Movement Ecology 12, 29.
| Crossref | Google Scholar |
Gigliotti L, Dietsch A (2014) Does age matter? The influence of age on response rates in a mixed-mode survey. Human Dimensions of Wildlife 19(3), 280-287.
| Crossref | Google Scholar |
Graefe A, Mowen A, Covelli A, Trauntvein N (2011) Recreation participation and conservation attitudes: differences between mail and online respondents in a mixed-mode survey. Human Dimensions of Wildlife 16, 183-199.
| Crossref | Google Scholar |
Herzog AR, Rodgers WL (1988) Age and response rates to interview samples survey. Journal of Gerontology 43, S200-S205.
| Crossref | Google Scholar | PubMed |
Im E, Cheeb W, Bender M, Cheng C, Tsai H, Kang NM, Lee H (2005) The psychometric properties of pen-and-pencil and internet versions of the midlife women’s symptom index (MSI). International Journal of Nursing Studies 42, 167-177.
| Crossref | Google Scholar | PubMed |
Johnson MS, Vanessa VM, Bryne J (2024) Addressing fraudulent responses in online surveys: Insights from a web-based participatory mapping study. People and Nature 6, 147-164.
| Crossref | Google Scholar |
Kaldenberg DO, Koenig HF, Becker BW (1994) Mail survey response rate patterns in a population of the elderly: does response deteriorate with age? The Public Opinion Quarterly 58, 68-76.
| Crossref | Google Scholar |
Kaplowitz MD, Hadlock TD, Levine R (2004) A comparison of web and mail survey response rates. Public Opinion Quarterly 68, 94-101.
| Crossref | Google Scholar |
Kelfve S, Kivi M, Johansson B, Lindwall M (2020) Going web or staying paper? The use of web-surveys among older people. BMC Medical Research Methodology 20(1), 252.
| Crossref | Google Scholar | PubMed |
Kiernan NE, Kiernan M, Oyler MA, Gilles C (2005) Is a web survey as effective as a mail survey? A field experiment among computer users. American Journal of Evaluation 26, 245-252.
| Crossref | Google Scholar |
Kuriakose N, Robbins M (2016) Don’t get duped: fraud through duplication in public opinion surveys. Statistical Journal of the IAOS 32(3), 283-291.
| Crossref | Google Scholar |
Kwak N, Radler B (2002) A comparison between mail and web surveys: Response pattern, respondent profile, and data quality. Journal of Official Statistics 18, 257-273.
| Google Scholar |
Laborde LP, Rohwer FC, Kaller MD, Reynolds LA (2012) Contrasts of waterfowl hunter surveys: open web and random mail surveys produce similar policy results. In ‘Proceedings of the Annual Southeast Association of Fish and Wildlife Agencies’. vol. 66, pp. 140–145. (Southeast Association of Fish and Wildlife Agencies)
Lawlor J, Thomas C, Guhin AT, Kenyon K, Lerner MD, Drahota A (2021) Suspicious and fraudulent online survey participation: Introducing the REAL framework. Methodological Innovations 14(3), 20597991211050467.
| Crossref | Google Scholar |
Leone EH, Bankovich B, Morales N, Sauby K, Shields R (2022) Response rate and content differences between web and mail surveys. Wildlife Society Bulletin 46, e1292.
| Crossref | Google Scholar |
Lesser VM, Yang DK, Newton LD (2011) Assessing hunters’ opinions based on a mail and a mixed-mode survey. Human Dimensions of Wildlife 16, 164-173.
| Crossref | Google Scholar |
Loomis DK, Paterson S (2018) A comparison of data collection methods: mail versus online surveys. Journal of Leisure Research 49(2), 133-149.
| Crossref | Google Scholar |
Lozar Manfreda K, Bosnjak M, Berzelak J, Haas I, Vehovar V (2008) Web surveys versus other survey modes: a meta-analysis comparing response rates. International Journal of Market Research 50, 79-104.
| Crossref | Google Scholar |
Manfredo MJ, Berl REW, Teel TL, Bruskotter JT (2021) Bringing social values to wildlife conservation decisions. Frontiers in Ecology and the Environment 19(6), 355-362.
| Crossref | Google Scholar |
Marshall AL, Leslie ER, Bauman AE, Marcus BH, Owen N (2003) Print versus website physical activity programs: a randomized trial. American Journal of Preventive Medicine 25, 88-94.
| Crossref | Google Scholar | PubMed |
McCann NP, Walberg EM, Forester JD, Schrage MW, Fulton DC, Ditmer MA (2021) Integrating socioecological suitability with human–wildlife conflict risk: case study for translocation of a large ungulate. Journal of Applied Ecology 58, 2810-2820.
| Crossref | Google Scholar |
Pealer LN, Weiler RM, Pigg RM, Jr, Miller D, Dorman SM (2001) The feasibility of a web-based surveillance system to collect health risk behavior data from college students. Health Education and Behavior 28, 547-559.
| Crossref | Google Scholar | PubMed |
Peterson CC, Messmer TA (2010) Can public meetings accurately reflect public attitudes toward wildlife management? Journal of Wildlife Management 74, 1588-1594.
| Crossref | Google Scholar |
Petchenik J, Watermolen DJ (2011) A cautionary note on using the internet to survey recent hunter education graduates. Human Dimensions of Wildlife 16, 216-218.
| Crossref | Google Scholar |
Pratt-Chapman M, Moses J, Arem H (2021) Strategies for the identification and prevention of survey fraud: data analysis of a web-based survey. JMIR Cancer 7(3), e30730.
| Crossref | Google Scholar | PubMed |
Quinn K (2010) Methodological considerations in surveys of older adults: technology matters. International Journal of Emerging Technologies and Society 8, 114-133.
| Crossref | Google Scholar |
Rodriguez HP, von Glahn T, Rogers WH, Chang H, Fanjiang G, Safran DG (2006) Evaluating patients’ experiences with individual physicians: a randomized trial of mail, internet, and interactive voice response telephone administration of surveys. Medical Care 44, 167-174.
| Crossref | Google Scholar | PubMed |
Saleh A, Bista K (2017) Examining factors impacting online survey response rates in educational research: perceptions of graduate students. Journal of MultiDisciplinary Evaluation 13(2), 63-74.
| Google Scholar |
Schaefer DR, Dillman DA (1998) Development of a standard E-mail methodology: results of an experiment. Public Opinion Quarterly 62, 378-397.
| Crossref | Google Scholar |
Schlosser S, Mays A (2017) Mobile and dirty: does using mobile devices affect the data quality and response process of online surveys? Social Science Computer Review 36(2), 212-230.
| Crossref | Google Scholar |
Schroeder SA, Fulton DC, Cornicelli L, Cordts SD, Lawrence JS (2019) Clarifying how hunt-specific experiences affect satisfaction among more avid and less avid waterfowl hunters. Wildlife Society Bulletin 43(3), 455-467.
| Crossref | Google Scholar |
Sexton NR, Miller HM, Dietsch AM (2011) Appropriate uses and considerations for online surveying in human dimensions research. Human Dimensions of Wildlife 16, 154-163.
| Crossref | Google Scholar |
Stedman RC, Connelly NA, Heberlein TA, Decker DJ, Allred SB (2019) The end of the (research) world as we know it? Understanding and coping with declining response rates to mail surveys. Society and Natural Resources 32, 1139-1154.
| Crossref | Google Scholar |
Stedman RC, Connelly NA, Lauber TB (2024) Understanding hunter support for early successional habitat management. Human Dimensions of Wildlife 29(1), 72-86.
| Crossref | Google Scholar |
Tourangeau R, Sun H, Yan T, Maitland A, Rivero G, Williams D (2018) Web surveys by smartphones and tablets: effects on survey responses. Social Science Computer Review 36(5), 542-556.
| Crossref | Google Scholar |
U.S. Census Bureau (2022) Types of internet subscriptions in Illinois. Available at https://data.census.gov/ [accessed 8 August 2024]
Vaske JJ (2011) Advantages and disadvantages of internet surveys: introduction to the special issue. Human Dimensions of Wildlife 16(3), 149-153.
| Crossref | Google Scholar |
Vaske JJ, Roemer JM (2013) Differences in overall satisfaction by consumptive and nonconsumptive recreationists: a comparative analysis of three decades of research. Human Dimensions of Wildlife 18(3), 159-180.
| Crossref | Google Scholar |
Vaske JJ, Fedler AJ, Graefe AR (1986) Multiple determinants of satisfaction from a specific waterfowl hunting trip. Leisure Sciences 8(2), 149-166.
| Crossref | Google Scholar |
Vaske JJ, Miller CA, Williams BD, Walberg E (2024) Declining hunter and trapper mail survey response rates between 1976 and 2020 in Illinois. Human Dimensions of Wildlife 30(3), 352-361.
| Crossref | Google Scholar |
Waggoner PD, Kennedy R, Clifford S (2019) Detecting fraud in online surveys by tracing, scoring, and visualizing IP addresses. Journal of Open Source Software 4(37), 1285.
| Crossref | Google Scholar |
Wardropper CB, Dayer AA, Goebel MS, Martin VY (2021) Conducting conservation social science surveys online. Conservation Biology 35, 1650-1658.
| Crossref | Google Scholar | PubMed |
Weigold A, Weigold IK, Russell EJ (2013) Examination of the equivalence of self-report survey-based paper-and-pencil and internet data collection methods. Psychological Methods 18(1), 53-70.
| Crossref | Google Scholar | PubMed |
Wolfe EW, Converse PD, Oswald FL (2008) Item-level nonresponse rates in an attitudinal survey of teachers delivered via mail and web. Journal of Computer Mediated Communication 14, 35-66.
| Crossref | Google Scholar |
Wright G (2015) An empirical examination of the relationship between nonresponse rate and nonresponse bias. Statistical Journal of the IAOS 31(2), 305-315.
| Crossref | Google Scholar |
