Site MenuSurvey of Electors – Whitby–Oshawa / Yellowhead By-elections
Appendix A: Detailed Methodology
On behalf of Elections Canada, Nielsen conducted a random telephone survey with 1,503 eligible electors in the electoral districts of Whitby–Oshawa in Ontario and Yellowhead in Alberta after the November 17, 2014 by-elections. The objective of the study was to evaluate the electors' opinions, attitudes and knowledge of the agency's services and their experience with the electoral process.
Only eligible voters, Canadian citizens 18 years of age or older on the day of polling and who had been residing in one of the ridings between October 15 and November 17, 2014, were invited to participate. Below is a detailed methodology of this project.
Sampling Strategy
Elections Canada was looking for a survey that was representative of eligible voters in both ridings. A stratified random sampling technique based on age and gender was applied and resulted in a similar composition on both locations. The goal was to survey a minimum of 750 eligible electors from the general populations on each electoral riding.
| Quota group | Desired # of completes | |
|---|---|---|
| Yellowhead | Whitby–Oshawa | |
| Male 18 - 34 | 75 | 75 |
| Male 35 - 54 | 150 | 150 |
| Male 55 and older | 150 | 150 |
| Female 18 - 34 | 75 | 75 |
| Female 35 - 54 | 150 | 150 |
| Female 55 and older | 150 | 150 |
| Total # of completes | n=750 | n=750 |
The final result was similar to the actual population distribution in both ridings and only minor adjusts were made via weighting. Below is the actual respondents' distribution.
| Quota group | # of completes | |
|---|---|---|
| Yellowhead | Whitby–Oshawa | |
| Male 18 - 34 | 107 | 102 |
| Male 35 - 54 | 153 | 154 |
| Male 55 and older | 126 | 106 |
| Female 18 - 34 | 101 | 103 |
| Female 35 - 54 | 145 | 165 |
| Female 55 and older | 118 | 124 |
| Total # of completes | n=750 | n=753 |
Questionnaire Design
Elections Canada provided a draft questionnaire which was used in previous waves of research and worked together with Nielsen team to create a final questionnaire that would maximize the project objectives. Although some changes were made, most questions were maintained in order to allow for comparisons across elections and by-elections.
The final questionnaire was composed of 45 questions and 13 demographic questions and took an average of approximately 12 minutes to complete.
Pre-Test
Nielsen conducted a pre-test using random digit dialing (RDD) to call residents of Whitby–Oshawa and Yellowhead ridings on the evening of December 17 from 5:30 pm onwards. The objective of the pre-test was to observe the respondents' experience during the survey, look for any inconsistencies or wording issues that may confuse the respondent and assess overall functionality of the survey instrument.
A total of 20 interviews were observed throughout the evening. A pre-test report was presented to the client group one day later and once EC reviewed the results, only minor changes were made to the questionnaire and therefore, results from the pretest were used in the final data.
Survey Launch and Data Collection
Interviews were administered by telephone between December 17 and 30, 2014, using random digit dialling technology and Confirmit Horizons software. Potential participants were asked qualification questions before being invited to participate in the survey. Respondent qualification was determined using a series of questions about citizenship status, age and residency of the respondent.
Interviews lasted an average of 12.04 minutes. A total of 1,503 valid survey completions were obtained. The margin of error associated with the combined sample size for this survey is ±2.5% while for the two individual ridings is ±3.6% 19 times out of 20 (95% confidence).
Weighting Scheme
At the completion of data collection, the data was weighted to reflect the information from the 2011 census. We used a standard procedure for calculating weighting factors, based on established methodological standards. This procedure involved calculating the actual population within each segment (in this case age and gender within region), and the true proportion of the sample that would fall into each segment if the survey were conducted on strictly a random basis (based on the most current population statistics available). Into this number was divided the actual segment subsample to produce a weighting factor that was then used to "weight" the data for that segment.
In both ridings there was a surplus of respondents aged 55 and older and an underrepresentation of respondents aged 18 to 34. The table below presents the weights applied according to the stratification plan.
| Whitby–Oshawa | # Population | % Population | # in Survey | % in Survey | WEIGHT |
|---|---|---|---|---|---|
| Males 18-34 | 15,030 | 13.56% | 67 | 8.90% | 1.524542001 |
| Females 18-34 | 15,155 | 13.68% | 74 | 9.83% | 1.391808347 |
| Males 35-54 | 22,615 | 20.41% | 152 | 20.19% | 1.01113285 |
| Females 35-54 | 24,235 | 21.87% | 152 | 20.19% | 1.08356421 |
| Males 55+ | 15,570 | 14.05% | 154 | 20.45% | 0.687104998 |
| Females 55+ | 18,195 | 16.42% | 154 | 20.45% | 0.802946399 |
| TOTAL | 110,800 | 100% | 753 | 100% | 1 |
| Yellowhead | # Population | % Population | # in Survey | % in Survey | WEIGHT |
|---|---|---|---|---|---|
| Males 18-34 | 10,740 | 14.30% | 63 | 8.40% | 1.701812097 |
| Females 18-34 | 10,080 | 13.42% | 74 | 9.87% | 1.359805167 |
| Males 35-54 | 15,325 | 20.40% | 151 | 20.13% | 1.013144545 |
| Females 35-54 | 14,550 | 19.37% | 153 | 20.40% | 0.949334878 |
| Males 55+ | 12,590 | 16.76% | 153 | 20.40% | 0.821451967 |
| Females 55+ | 11,845 | 15.77% | 156 | 20.80% | 0.757981038 |
| TOTAL | 75,130 | 100% | 750 | 100% | 1 |
Impact of weighting
The table below outlines the unweighted and weighted counts by age and gender in both ridings. It can be noted that the biggest adjustments were required to males in 18-34 age group in both ridings, which is a common observation among random probability surveys conducted in Canada via landline interviewing. The weighting scheme accounts for this shortfall and adjusts the final sample so that it represents the general population distribution for both regions.
| Riding | Sample group | Sample composition | |
|---|---|---|---|
| Unweighted | Weighted | ||
| Whitby–Oshawa | Male 18-34 | 67 | 102 |
| Male 35-54 | 152 | 154 | |
| Male 55+ | 154 | 106 | |
| Female 18-34 | 74 | 103 | |
| Female 35-54 | 152 | 165 | |
| Female 55+ | 154 | 124 | |
| Yellowhead | Male 18-34 | 63 | 107 |
| Male 35-54 | 151 | 153 | |
| Male 55+ | 153 | 126 | |
| Female 18-34 | 74 | 101 | |
| Female 35-54 | 153 | 145 | |
| Female 55+ | 156 | 118 | |
| TOTAL | 1,500 | 1,500 | |