3.4 Practical questions related to Information Management

TABLE OF CONTENTS

• 3.4.1 I need to build a survey to monitor my program activities - what are the key things to keep in mind?
  • Collecting only what is needed
  • Preliminary survey testing in the field
  • Cultural sensitivity
  • Survey format
• 3.4.2 I need to build a database - how should I get started?
• 3.4.3 Automatic dashboards updated in real time: what are the first steps to know?
• 3.4.4 How can I build a map of my data?
• 3.4.5 Are IM systems also relevant for qualitative data / information?
• 3.4.6 Data breach, risk 0? Why invest time and money in data protection?
• 3.4.7 What are the benefits of effective IM systems?
• 3.4.8 How can IM support data visualization for decision making & reporting?
• 3.4.9 Does MDC improve my program data’s quality?
• 3.4.10 What are the Mobile Data Collection tools: which one is best for my team?
• 3.4.11 How does Kobo improve data collection and IM?

---

3.4.1 I need to build a survey to monitor my program activities - what are the key things to keep in mind?

Surveys are often the most common measurement tool for humanitarian and development actors to connect with beneficiaries and target communities. Surveys produce quantitative data that can then be analyzed to understand the context, determine appropriate project activities, and understand the impact of programs. Let’s look at a few of the key aspects here.

Collecting only what is needed

The first point of consideration in designing surveys is to ensure that only the minimum and essential information is collected, meaning that all data coming from surveys should be used. Before creating the survey, determine the data collection needs through developing an analysis plan, which should then be used to guide the development of specific questions.

A common mistake in designing data collection tools is that they are seen as a separate issue from developing analysis outputs. Developing an analysis plan will counteract this risk; it will help you think through the links between what information is needed, the specific survey questions which will collect this data, who will need to answer the questions, and how each question will be analyzed to provide the information. For details on developing an analysis plan (and how it relates to developing a database), please see page 4 of the ACAPS guide, found here.

Preliminary survey testing in the field

Another key consideration in developing surveys is that there is always the need to field test the survey to ensure its relevance and appropriateness in the context. Survey questions can sometimes be unclear to both enumerators as well as respondents. By testing, you can reduce the uncertainty among both groups and promote better quality data. Field testing requires the following steps:

1. Extensive training of data collection teams (enumerators) on the survey questions. Review each question in turn and ensure that enumerators know the purpose as to why each question is being asked - refer back to the analysis plan!
2. If questions are unclear based on enumerators’ knowledge of the context, ensure that they are edited before deployment. Therefore, it can be recommended to directly involve local teams during the design phase of the survey questions wherever it is possible.
3. Field test the survey among a small sample of the target respondents.
4. Review the data that was collected during the pilot – Were any responses very unclear or unlikely? Did many respondents choose ‘don’t know’ to multiple choice questions?
5. Meet with the field teams and review their perceptions of how the pilot went – Were any questions unclear to them or the respondents? Were any questions sensitive to the beneficiaries and either not asked, or asked in a more culturally appropriate manner?
6. Adapt the survey in accordance with the information received in relation to each specific question.

Cultural sensitivity

Piloting surveys is particularly important when surveys have been translated, which can sometimes lead to a change in meaning, lack of clarity, or potentially remove nuance in accordance with the local context. Please review the following paper by Indikit and Translators Without Borders for guidance on Localizing and Translating Survey tools (found here).

Survey format

Another key consideration in developing a survey is whether or not you plan on using mobile data collection (MDC) or a paper-based survey. To consider the advantages of MDC as well as different solutions, please refer to the question below: ‘What are some examples of Mobile Data Collection tools, and which one is right for my team?’.

For further details, including considerations focused on planning, question quality and format, please review the ACAPS 2012 Technical brief, found here.

3.4.2 I need to build a database - how should I get started?

A database is a “tool that stores data, and lets you create, read, update, and delete the data in some manner” (ACAPS, 2013, pp. 3). Therefore, if you have organized program data using paper or computer software, you already have a database. Databases offer efficiency to users though providing a storage facility where data can be retrieved and updated. In turn, databases facilitate analysis by structuring data in a way that is simple to conduct calculations.

Deciding how to ‘model’ (or structure) your database depends on how you plan on using the data. Key considerations for creating a simple database are:

• ensure data is easy to enter;
• ensure the database does not require specialist technical skills; and,
• ensure the database structures data so you can easily conduct analysis.

Often, Excel is used as the database software solution among humanitarian and development actors, particularly at the program-level. Despite being a spreadsheet application, Excel has the benefit of easily entering, storing and analyzing small amounts of data. Additionally, basic knowledge of the software is widespread. Excel has many functions to improve data quality at the entry, such as through data validation tools. Excel can also be used easily in the cloud, which improves greatly the experience of using and sharing sheets between colleagues (co-authoring). Moreover, Excel sheets that are stored in the cloud (for example using the organization’s Office365 or Google suite account) can be easily linked and create fully automated dashboards at zero extra cost.

For more detailed information on creating a database, ACAPS produced a guide in 2013, found here.

3.4.3 Automatic dashboards updated in real time: what are the first steps to know?

Some organizations have automatic dashboards of their activities that are updated in real time. What are the first steps to doing the same?

Dashboards are a data visualization tool that usually aim to be easily understandable to a non-technical audience. Dashboards show visual representation of information produced through data analysis in a user-friendly manner to assist with the comprehension of information coming from the data.

After project data needs are defined, data have then to be collected, cleaned and stored. Only with quality data through these processes can program data then be analysed and visualized. As seen on the data management cycle diagram below, dashboards and other forms of data visualization would fit into the later ‘Share and receive feedback’ section.

Program dashboards must not be seen in isolation of other processes required for effectively program data management systems. As such, the first steps to developing effective dashboards are to ensure that the prior steps of the data management are in place (for further details, review the question: “3.3.1 What are the first steps to improving Information Management within my team?)”.

There are many different dashboard solutions that can be chosen, including Power BI, Tableau, R Studio, or even Excel. Importantly, dashboards require regular updates and maintenance, meaning internal teams must be aware of the processes to manage the software of choice. Choosing the software solution should often be based on capacities available within the team, unless an organization is willing to invest in staff training.

Finally, regular updates of dashboards require streamlined processes for data collection, cleaning and storage, which will then need to feed into the dashboard for data analysis. Streamlining this process requires an automated system of data collection and cleaning, which then feeds into the dashboard for analysis/visualization.

3.4.4 How can I build a map of my data?

Mapping can often be the poor relation in terms of data management in the humanitarian and development sectors. While it can produce visually appealing and effective ways to share program information, it generally requires a considerable amount of time and a significant technical background, that can be hard for an organization to plan, except if mapping is really key to the better understanding of the organization’s activities.

If you want to stay on very limited but easy to set up desktop or online tools, you can consider Google Earth to start off with for the desktop approach, or tools like GogGoCarto for an online map to see if they answer your needs.

If a proper GIS system makes more sense as you need to have a constant geographical understanding of your activities (and you therefore have-internally or externally- the resources to set it up, maintain such a system and produce the maps), there are two tools that are the most used: QGIS (a free open source software) or ArcGIS (a proprietary advanced software). Both have their pros and cons and should therefore be evaluated based on the needs you have (advanced geospatial features, very beautiful map editing, etc.).

CartONG has produced a toolbox for the humanitarian and development sectors which aims to address both theoretical concepts as well as provide tutorials of useful tools in the practice of cartography and – for more advanced profiles – Geographic Information Systems (GIS) that you can access here (currently only in French). The approach is based on a set of concepts and tools that must be mastered to produce quality maps and spatial analyses.

3.4.5 Are IM systems also relevant for qualitative data / information?

Qualitative data provides information and knowledge that is often essential to understand a situation on the ground. A comprehensive understanding of field realities is necessary to maximize the benefits of humanitarian services. Therefore, qualitative data should also be incorporated alongside quantitative data into IM as much as possible, especially as it is too often put aside as considered longer to approach and analyze in our very “quantitative data driven” world.

The ways of incorporating qualitative data into IM systems are less clear than for quantitative data. For example, qualitative data does not lend itself as well to MDC, which allows for a high degree of automation with quantitative data. However, qualitative data can be used through methods such as informing the design of quantitative tools, or through ‘diving deeper’ into unexpected findings coming from quantitative data during different research processes (exploratory research, M&E, etc.).

Further, sharing quantitative findings with communities through community meetings or focus group discussions can also be an effective method for the collection of qualitative data, while also providing community feedback.

3.4.6 Data breach, risk 0? Why invest time and money in data protection?

There is very little risk of beneficiaries being affected by a data breach, so why should we invest time and money to implement data protection measures / put data protection in practice?

In the humanitarian and development sphere, NGOs collect and process personal data to perform activities, and therefore there is always risk of data breach, and such breaches have occurred before and will continue to occur. Moreover, in such environments where the rule of law may not be fully applied, “the protection of Personal Data of beneficiaries and staff is often necessary to safeguard their security, lives and work” (ICRC, Handbook on data protection in humanitarian action, pp 28-29).

As stated by ICRC, “Protecting individuals’ Personal Data is an integral part of protecting their life, integrity, and dignity” (Handbook on data protection in humanitarian action). New technologies have allowed for easier and faster processing of personal data, which in turn leads to concerns about intrusion into private lives.

Even with the best tools in place, we cannot ignore the risk of technical defects that could lead to breaches in data security, that could have considerable negative consequences on beneficiaries. More importantly, the ‘human factor’ is the most important consideration for potential data breaches. Human risks may include unconscious sharing of personal information, making it very important to sensitize all staff on responsible data and data protection issues. Therefore, it is essential for staff to understand key terms, such as Personally Identifiable Information (PII) and Sensitive data:

• Also called “direct identifiers”, Personally Identifiable Information (PII) is specific personal data that can directly identify the identity of a person. PII can include data such as a respondent’s name, address, or ID number.
• Sensitive data is data that can cause harm if disclosed or accessed without proper authorization. Examples may include data in relation to health, race or ethnicity, or affiliation to religious and political groups. Sensitive data could cause harm to a person or have a negative impact on an organization’s ability to carry out its activities.

Because humanitarian work is done in diverse contexts, the sensitivity of data and the appropriate safeguards must be determined on a contextual basis. An overview of existing guidance on the protection of personal data in humanitarian action can be found in ICRC’s 2020 Handbook on Data Protection and OCHA’s 2021 Data Responsibility Guidelines.

3.4.7 What are the benefits of effective IM systems?

Do the benefits of effective Information Management systems extend beyond facilitating easier donor reporting?

Information management is intended to facilitate quality data for analysis that allows program teams to make evidence-based decisions. Doing so includes continuous input of program-based M&E data alongside analysis and visualization.

A side-benefit of effective IM systems is ease of reporting program data to donors, but not the purpose itself. Centralized IM systems can remove reporting challenges that exist as a result of multiple datasets in different folders, different staff having their own tracking systems that don’t always correspond, or other common issues in the sector.

In sum, effective and centralized IM creates the evidence base needed to improve NGO programming, while also reducing time-spent focusing on donor reporting.

3.4.8 How can IM support data visualization for decision making & reporting?

More precisely, how can Information Management support and automate data visualization to help program decision making and reporting?

Data visualization is the visual representation of data analysis, for example through tables, charts or graphs. Effective information management will allow for visualizations of program data. Program data should be stored in a database, through which data visualization can be automated (for example, through the use of Excel). When new inputs of program data are added to the database (for example through programming monitoring) the visualizations can adapt automatically. For example, rapid monitoring of activities can be very effective through the use of MDC systems directly connected with dashboards.

With regards to data visualization, it is important to note that often ‘easier is better’ when it comes to selecting the tools. Often organizations in the aid sector experience high staff turnover, and complex, advanced software can take extended time and effort for training (such as R, SPSS or Stata). Often tools should be adopted that require little time training, such as Excel (Tdh & CartONG, Data Visualization Toolkit, pp. 25.

For detailed guidance on data visualization, please see Tdh and CartONG’s toolkit, found here.

3.4.9 Does MDC improve my program data’s quality?

Here we are looking at how Mobile Data Collection can improve the quality of my program data. What are the experiences of other NGOs?

Data quality can be defined as the ability with which data can be used to make decisions, thereby incorporating multiple concepts such as data relevance, accuracy, timeliness, accessibility and comparability (for further details see the ‘Information management Beginner’s Glossary’, here).

Mobile Data Collection (MDC) will never replace proper critical thinking when conceiving your forms and database, but - when done correctly - it has the ability to improve data quality (compared to paper-based systems) through addressing accuracy, timeliness and comparability. Data accuracy can be improved through imbedded features within MDC forms, such as choosing different question types (select one, select multiple, etc.), skip logic (questions are only asked if relevant to the respondent based on prior responses), and constraints (answers to questions are limited based on criteria established as realistic by the researchers).

In addition, comparability of data can be improved through MDC by standardizing the way in which questions are asked across multiple forms. For example, the same survey or questions can be easily duplicated for other rounds of data collection. Doing so can make sure that indicators are comparable over time – which is needed for quantitative measurements of impact within M&E – or across different humanitarian activities.

Finally, MDC can improve the timeliness of data in terms of data cleaning and aggregation. Without MDC, enumerators or other staff have to spend long periods of time manually entering data into a database, i.e. reviewing paper survey forms and inserting each response directly into a spreadsheet. Manual data entry is also prone to human error, so removing this step is another way in which MDC can improve data quality.

MDC can be very easily linked to automated monitoring dashboards. Thus monitored can be conducted in real-time from the start, which allows teams to adjust processes rapidly, if/when necessary.

For getting started and a detailed guidance on each step related to MDC, please see Tdh & CartONG’s toolkit, found here.

To understand the experience of another NGO in their transformation to MDC, please see the Lessons Learnt Paper from Tdh and CartONG, here.

3.4.10 What are the Mobile Data Collection tools: which one is best for my team?

There is a breadth of mobile data collection (MDC) solutions with different strengths and limitations, and as such, it is important to review different solutions before investing time and energy into implementation. For a detailed overview of common MDC tools, please review CartONG’s 2021 MDC solution benchmarking report, here.

Different ways of categorizing MDC solutions include whether or not the software is more generic, or designed for further, detailed functionality in specific sectors (see diagram below).

When considering which MDC solution to choose, the strengths and weaknesses should be reviewed in relation to the needs of your organization. For your assistance, CartONG’s 2021 MDC solution benchmarking report focuses on the following aspects of different solutions:

• Organizational management – Is there a centralized management system of users and surveys?
• User experience – Is the tool intuitive and easy to adopt?
• Data quality – Are there dedicated features such as constraints and skip logics?
• Data protection – Does the solution facilitate GDPR compliance through dedicated features to flag and limit access to personal identifiable and sensitive data?
• Case management – Are features available that facilitate case management and data collection / review over-time?

3.4.11 How does Kobo improve data collection and IM?

KoBoToolbox is a mobile data collection (MDC) platform that is a free tool particularly developed for humanitarian actors in emergencies and difficult field environments. It is built on the very active open-source OpenDataKit (ODK) ecosystem that has been a game changer in terms of mobile data collection since the first smartphones came out more than 10 years ago. It is. It can be used for various assessments, monitoring, and other data collection activities. Organizations can decide to use the OCHA server (dedicated to humanitarian organizations), the HHI server (for researchers/other aid actors) or to host their own Kobo server. Find out more here on the first two options.

Like any MDC platform, Kobo has the ability to greatly improve program data quality and therefore the overall information management system (see question above: “Does MDC improve my program data’s quality?)”. Using Kobo should be based on the project’s information needs defined in the M&E and analysis plans. MDC can be an enabler of increasing efficiency in your data analysis, which will further improve your IM system (see page 24 of Tdh and CartONG’s Lessons Learnt Paper on implementing MDC, here).

Kobo is one of many potential MDC solutions, all of which have strengths and limitations. You can see as example Terre des homme’s rationale in terms of the different MDC tools they are using depending of their different purposes on the diagram on page 18 of the Lessons Learned paper, here.

For a more detailed overview of different MDC tools, please review CartONG’s 2021 MDC solution benchmarking report, here.