Appendix B: Pretest Report – Survey of Candidates of the 43rd Federal General Election

Submitted to: Evelyne Morrissette
Research Analyst
Policy, Planning and Public Affairs

Elections Canada
30 Victoria Street
Gatineau, Quebec
K1A 0M6

Ekos Research Associates Inc.
October 30, 2019

The project began with a thorough review of the draft survey instrument provided by Elections Canada in October 2019. EKOS Research provided input on possible changes using the draft questionnaire. As a result, the survey was programmed and thoroughly reviewed with minor additional changes made to programming and branching logic. Test interviews were conducted on October 22, when 26 interviews (13 in French and 13 in English) were completed at an average time of 26.5 minutes overall excluding several lengthy outliers (24 minutes on average in English and 27 minutes on average in French). Two cases were also completed online at roughly the same average length.

The project manager carefully reviewed the survey data collected from the test for data quality and appropriate branching logic, and also reviewed a sample of the audio recordings in order to assess the need for further changes to the questionnaire. The project manager and client representatives each listened to a sample of recorded interviews and discussed a number of changes to the questionnaire. Several instructions for interviewers were clarified as a result of discussions about the results. A number of changes were also suggested to increase clarity and provide additional instructions to interviewers. Given the length of the initial pretest interviews, wording was streamlined in a few instances. Changes were made to the survey instrument and implemented in the programming. Call disposition results from the test were as follows:

Call Outcome	Count
Invalid Number	2
Callback	180
Appointment	17
Incomplete Refusals	0
Refusal	4
Preference to complete online	4
Complete	26
Total	233

The revised questionnaires are provided in Appendix C. The survey collection began on October 25. Based on the length of the survey (averaging 25 minutes on the telephone and online), the decision was also made to emphasize the availability of the online version as much as possible to increase the efficiency of the survey collection and maximize flexibility for survey respondents. On October 28, there was a careful review of the overall frequencies of responses and a comparison of the responses obtained online versus those recorded by telephone. In several questions, added instructions were applied to the telephone version in an effort to increase consistency of results between the two modes. It was also noted that in a number of instances (typically when a satisfaction scale was used), results were more positive during a telephone interview than when respondents used the online survey. Full analysis of the survey results will be thoroughly reviewed in terms of a comparison by mode to examine sample and mode effects. The pretest cases will be included in the final sample for analysis, with some minor adjustments based on changes as a result of the test.

• Previous
• Table of Contents
• Next