3.1.2 Alternatives to survey-type data collection

TABLE OF CONTENTS

• Remote alternatives to survey-type data collection
  • Summary of solutions and glossary
  • Glossary
  • Prerequisites and Implications
    • Administration by interviewer
      • Telephone Interview
      • CATI - computer-assisted telephone interviewing
    • Self-administration
      • SMS Survey
      • Survey by USSD
      • Smartphone web survey
      • Interactive Voice Response (IVR)

---

Remote alternatives to survey-type data collection

Summary of solutions and glossary

Even if the face-to-face interaction of a regular survey cannot be completely replaced, the use of one of the following communication methods makes it possible to overcome an information deficit or to triangulate information received:

Glossary

USSD: Real-time or instant messaging type mobile phone service, without the possibility of saving or forwarding messages (unlike the characteristics of “normal” SMS messages. Interactive USSDs are message exchange sessions, in a global time frame defined in advance by the mobile network operator (negotiable). It is the exchange session that will be billed, not each message sent (as is the case for SMS).

For some of these alternatives (see below), you may also need to solicit and contract with one or more MNOs and/or a mobile aggregator. Here is the difference between these two types of organizations:

Mobile Network Operator (MNO): Organization providing telecommunication services. For example: Orange or MTN.

Mobile Aggregator: An organization that incorporates multiple mobile network operators and provides “unified” access to the different communication services of the mobile network operators(MNOs) through their aggregation application. Note that there are different types of aggregators (API aggregator, money transfer aggregator…) and that some are country specific, especially for USSDs. The most common are Twilio, Vonage - formerly Nexmo and Tokbox, Clickatell, Cellulant, InfoBip, Africa’s Talking.

Prerequisites and Implications

For all the identified alternatives, it is essential that the population has, at the very least access to a telephone service, which can therefore be a hindrance in certain contexts. The respondent must have access to a mobile phone and be able to use it. For some of these alternatives, access to the Internet is also essential.

This introduces significant and difficult to circumvent biases in the profiles of survey respondents, which require that the results be analyzed with a certain degree of reservation (it is imperative to document the identified biases).

1. The penetration of mobile phones is not homogeneous in all territories, for various reasons (network coverage, population density, culture, etc.).
2. In many countries, telephone owners are proportionally more likely to be young men. It may therefore be difficult to access certain audiences (depending on gender and age).
3. Having a phone (and being able to charge it) or not impacts the representativeness of the results in terms of vulnerability. The phone model also plays a role: between a mobile phone and a smartphone, the profile of the owner will not be the same.
4. There is an inherent bias in technology. Some communities may have a negative perception of them (e.g., lack of trust, seen as tools for spying, etc.). In some cases it might be necessary to work on such negative perceptions with the communities prior to the data collection.
5. Finally, in relation to the previous points and because some alternatives based are on written exchanges, the bias of illiteracy may appear.

Moreover, adapting one’s collection to these alternatives implies:

• To review the content of surveys using new modalities: shortening the questionnaire (15-20 minutes on the phone seems to be a maximum), prioritization and simplification of questions possible reformulation of open-ended questions into simpler or multiple-choice questions (for SMS or IVR), impossibility of collecting certain data such as observations (cleanliness of a latrine for example) or questions too sensitive to be asked over the phone, etc. It is therefore essential to think about the indicators that one is seeking to analyze, to prioritize them, and to revise the questionnaire accordingly. The test phase of the said questionnaire becomes even more important.

• Another fundamental technical aspect is the translation(s) of the questionnaire which must be carried out prior to the data collection for SMS and IVR surveys. It is also necessary to ensure that the interviewers/administrators are fluent in the local language, in order to respond to any remarks.

• A higher non-response rate should be incorporated when designing the sampling design, it is likely that the non-response rate will increase when using a telephone rather than face-to-face data collection (due to dead batteries, network outage, unwillingness to answer etc.). More generally, the non-response or refusal rate is an important point to monitor in order to assess the adjustment of the data collection.

• Human resources are also to be considered: does the data collection tool involve recruiting interviewers? Does the process need to be automated (software, voice message) and therefore require specialized data management profiles? And if so, are such profiles available? Even if surveys are conducted by telephone, it is essential that trust can be established between the respondent and the interviewer (gender, age and language criteria are crucial).

• Using new tools means thinking about budget, which is not necessarily negligible. Do the surveyed populations have telephones? Should phones/SIM cards be distributed? How should respondents be compensated (phone credit/SIM cards)? What is the most appropriate technological solution (depending on its business model: in terms of phone credit, number of messages sent)?Does the organization have the budget to meet these expenses?

Administration by interviewer

Telephone Interview

Telephone interviewing is suitable for surveys that have been previously carried out on paper or on a mobile device (Mobile Data Collection - MDC) and where the automation of call management is not essential (low volume of people to call).

Technological solutions: The same as used previously, i.e. paper forms or ideally mobile data collection solutions such as:

• KoBoToolbox: via web forms (Enketo)
• SurveyCTO: via the universal web interface
• ONA: via web forms (Enketo)
• CommCare: via WebApps

Recommendations:

For relatively short surveys, or surveys with few respondents and especially for qualitative surveys. Surveys are carried out as before in MDC, but via telephone after adaptation of the questionnaire.

Preferable if you are familiar with MDC-type data collection in particular, if you do not need to invest in specific CATI software.

Do not underestimate the change in practice generated by telephone surveys, even for experienced interviewers - remember to train them well, debrief them at the beginning and check the quality of the data collected regularly.

CATI - computer-assisted telephone interviewing

CATI is suitable for scaling up a telephone survey: the software automatically organizes the schedule of calls and callbacks, it excludes non-respondents (after so many non-responses, if there are wrong numbers…) and it makes it possible to record calls.

Technological solutions: these can be either:

• The same mobile data collection solutions as the CATI “starter kit” module of SurveyCTO (Computer-Assisted Telephone Interviewing), although few offer these solutions,
• Specific solutions dedicated to the sector such as Geopoll,
• Software dedicated to call centers such as Voxco CATI.

Recommendations:

For surveys involving a very large number of individuals with many interviewers and whose management needs to be automated.

Preferable if you can adapt quickly (survey and staff) and plan that future surveys can be carried out in this way afterwards (so that the investment is justified).

Self-administration

Self-administration of the questionnaire has two major implications:

• First, the honesty of the answers may increase, and thus the quality as well (because when respondents do not interact directly with an interviewer, they are pressured to answer honestly, and are less likely to adapt their answers to what the interviewer would like to hear);

• Secondly, the quality of the answers may on the contrary diminish (because it is not possible to interact or to check that the person has understood the question - and thus to ensure that the answer is genuine and not a random answer).

SMS Survey

Technologies: Open source solutions exist but they require technical support (for a fee).

• FrontlineSMS: free open source technology in desktop version but cloud version (highly recommended) a paying service,
• RapidPro (former RapidSMS and TexIt){:target=”_blank”}: open source solution but requiring dedicated hosting, and even technical support. Allows the design of dedicated modules such as Ureport.

A large number of companies specialising in the Aid sector also offer this type of service:

• Geopoll,
• Viamo (formerly VOTO),
• Echo Mobile,
• EngageSpark,
• Telerivet - charged from 50 messages per day and 50 contacts.

Finally, some mobile data collection solutions offer SMS surveys as an additional module that is quite easy to handle (to be preferred if you already use these solutions):

• CommCare allows you to create SMS surveys.

A comparison of SMS solutions has been updated in 2019: http://impacttrackertech.kopernik.info/digital-data-platform/ (Kopernik - International Plan).

Recommendations:

For relatively short and simple surveys of large numbers of people with sufficient literacy skills. Surveys need to be adapted. Consideration should be given to compensating respondents either by direct reimbursement (airtime, airbundle) or by the organization bearing the cost through the technical provider (possible in many cases). You will most likely need support in setting up this type of system.

Survey by USSD

USSDs are sessions for the exchange of messages, with a response time set in advance by the mobile network operator. This is the system generally used to top up credit on a phone.

Technologies: Providers of these solutions are generally only available in certain countries (the following solutions work in many African countries / ask for more information if you need it).

• inTarget: partner company to deploy USSD studies,
• Echo Mobile: partner company to deploy USSD studies to communities in Africa.

Recommendations:

For short and simple or easily adaptable surveys with a literate audience, and if you have the budget to partner with an aggregator. The main advantage of this tool, compared to SMS, is that it does not store messages, thus protecting sensitive responses. You will most likely need support for this type of system.

Preferable if your survey is short/modular, and if you have the funding for a partnership with an aggregator.

Smartphone web survey

Technologies: The same as for MDC, only remote: KoBoToolbox, SurveyCTO, CommCare: via web forms sent by SMS, email, Whatsapp or other means of communication.

• Survey applications such as LimeSurvey, SurveyMonkey, Framaforms, etc.
• Geopoll- Monitoring and Evaluation Research in Developing Countries): partner company to deploy mobile web studies.

Recommendations:

Self-administration of questionnaires via a smartphone can be a solution in the case of targeted surveys among groups already trained or used to questionnaires (social or medical workers for example, local authorities, etc.). It is preferable for a group of literate respondents who are familiar with surveys and who have smartphones and internet access (Wifi in their business premises for example).

Interactive Voice Response (IVR)

Technologies:

• RapidPro (former RapidSMS and TexIt) open source solution (but requiring hosting oreven technical support) also makes the IVR,
• Echo Mobile - partner company to deploy IVR studies to communities in Africa,
• Telerivet - Professional mobile messaging made easy,
• EngageSpark - Send and Receive Automated Call and SMS Text Campaigns,
• Viamo - A global social enterprise improving lives via mobile.

A comparison of SMS solutions has been updated in 2019: http://impacttrackertech.kopernik.info/digital-data-platform/ (Kopernik - International Plan)

Recommendations:

For short and simple or easily adaptable surveys, and if you have the budget to partner with an aggregator. When respondents are unfamiliar with this type of “robotic” conversation, the rate of non-response or survey discontinuation may be high. See, for example, WFP’s feedback on DRC Interactive Voice Response mVAM. You will most likely need to be accompanied for this type of system.

Preferable if your survey is short/modular, and if you have the funding for a partnership with an aggregator.