RIWI's Technique for Polling Hard-to-Reach Populations

Some populations are inherently difficult to survey. As a result, their voices rarely figure into studies of public attitudes, financial or medical needs, or consumer preferences. The most overlooked populations include people without access to the Internet; immigrants, refugees, and persons internally displaced within their own countries; homeless persons; and people living in low-income, repressive, or war-torn countries.

Some organizations have made significant steps towards addressing this problem, and one such group is RIWI Corp., a publicly-traded company based in Toronto, Canada. To our great joy, RIWI has agreed to become an Azimuth Social Research partner, giving us access to a whole new range of international survey research capabilities.

RIWI’s Random Domain Intercept Technology (RDIT) invites web users worldwide to participate in surveys in their own language. RIWI bypasses national or sub-national web restrictions by leveraging its ownership, or temporary control, of hundreds of thousands of unused, expired, or abandoned web domains.

Many web users will stumble across one of these unused domains while surfing the Internet. Often, this stumble takes users to a “404” or similar type of error message, but when users run across one of RIWI’s sites, they are instead invited to take part in a quick, anonymous survey. RIWI’s system auto-detects the user’s geographic location, which allows them to offer the survey in a relevant language and to geolocate (when possible) the respondent’s general position.

RIWI’s method is useful in part because it circumvents government restrictions, as no other agency has a list of RIWI-controlled domains. The RIWI domain inventory, moreover, is constantly changing. As a result, RIWI’s survey work is particularly useful in countries such as China, where the government keeps tight control over its population’s access to Internet platforms, domains, and services.

RIWI’s methods also help researchers gain access to populations in regions of the world where law and order has broken down, or where the commercial or public landline infrastructure is spotty. Mobile phone penetration is sometimes high even in remote or conflicted area, and any user with Internet access has an equal chance of stumbling across one of RIWI’s domains.

Gathering data safely and securely is key. RIWI’s surveys are anonymous, and it neither gathers nor stores information that might identify respondents. Its geolocation feature does not identify specific respondents, and in any case is indicative of a general area, rather than a specific address.

There are drawbacks to RIWI’s methodology, of course, as there are for every survey method. The most important weakness, perhaps, is that RIWI can only reach people who already have Internet access. Although Internet penetration is high in many areas and growing in others, there are still populations with no or intermittent access. Web use, moreover, tends to be skewed towards younger males. RIWI compensates for the first by giving more statistical weight to the responses of older people and women, and address the second by further adjusting the weights to mimic what is already known about general population demographics.

Thus, for example, if the proportion of highly educated RIWI respondents from a specific country is greater than the real proportion as per census data, RIWI will “down-weight” the responses of educated respondents and “up-weight” those of the less educated. This procedure can be replicated for gender, ethnic makeup, geographic location, socioeconomic status, and more.

No survey weighting procedure is foolproof, and the gold standard remains careful, face-to-face surveys in which researchers knock on doors and religiously follow survey guidelines. In this method, researchers return again and again to houses where no one answers, and carefully select respondents by making exhaustive lists of each household’s residents and then using predetermined criteria to choose one to survey. The gold standard is expensive and sometimes impossible for reasons of safety or physical access, however.

Another problem with RIWI’s method is the response rate, which ranges from 13-17%. That means that over 80% of people who are presented with a RIWI survey don’t click on it. Response rates, moreover, tend to taper off after the first few questions; more people will answer questions 1, 2, and 3 than questions 9, 10, or 11. Low response rates of this sort are unfortunately not abnormal, however. In US telephone surveys, the typical response rate is well below 10%.

Response rates are of course higher when you knock on doors or sample from a list of pre-booked respondents who have already agreed to answer your questions, typically on the Internet. The door-knock method, which my fellow Azimuth advisors and I have used to great effect in countries where labor costs are lower, is one that I enjoy personally, as it offers endless opportunities for interesting conversations and forays into neighborhoods I wouldn't normally explore. My fellow Azimuth colleagues and I wrote our last book based on that method, and had a blast doing it (we worked with four other excellent Azimuth research partners on that project, including Team C Voter in India, LMS-CSA in Morocco, Data OPM in Mexico, and Practical Sampling International in Nigeria) .

Still, face-to-face door knocks are typically too expensive for most researchers in developed countries, so many have turned to using the prearranged panels of Internet users operated by survey companies such as YouGov, which fielded my own nationally representative poll of US residents in fall of 2018. (For a sample of articles based on that survey research, see this in the New York Times, another in Foreign Policy, and a third with the Washington Post.)

And yet, telephone, door knock or pre-arranged Internet panels are not always possible, and in these cases, RIWI offers a clear advantage.

You can read a detailed summary of RIWI’s methodology, strengths and weaknesses on pp. 88-91 of this recent World Bank report. In future blog posts, I’ll write more about individual RIWI studies, as I find them intriguing.

Azimuth Research Partner RIWI Corp tracks global public opinion trends, tests the effectiveness of messaging and advertisements, and measures the needs of vulnerable populations through its patented internet survey technology. RIWI collects no personally identifiable data. Its clients include G7 governments, the United Nations, World Health Organization, and many others.

Azimuth Advisors are experts in international survey research, among other skills. To learn more about Azimuth Social Research and how we can help your organization or project, please visit our Services page and email us at info@azimuthsocialresearch.com.