Equitable Data Collection: Designing for Your Audience

This is the second part of our series about equitable data collection. In the first part, we discussed the basic principles for how and why your should collect data. This next part covers important design considerations regarding language, accessibility, and structure when crafting forms, questionnaires, surveys, or any other tools that gather information from constituents.

Language

Baking inclusivity and accessibility into every part of your data collection process is key to ensuring your audience feels respected and informed, and that there are no barriers to answering your questions. This can be done by using inclusive language both in your data collection tools and in your messaging around your data collection efforts, conducting outreach and testing with members of your target audience so you can identify any issues early in the process, and following accessibility guidelines in your writing and formatting of questions.

Language is constantly evolving, and how people refer to themselves and like to be referred to is shifting as well. Do your research on the most appropriate terms for referring to groups of people and different identities, especially when referring to groups of people who have been historically underserved, underrepresented, and/or marginalized. There are many guides around inclusive language (like these examples from Hello Seven and the APA Style) that can help you use the most accurate and inclusive language possible.

We also recommend proactively reaching out to people in your target audience to pilot or test your survey or form and to find out what terminology and phrasing they use for themselves and what different terms mean to them. In all of your writing, be intentional about the level of specificity in which you describe people, focusing only on relevant characteristics, and be sensitive to preferred language. As a rule, call people what they call themselves. Piloting your data collection tools with a representative subset of your audience can help you understand if your messaging is clear, what terms your audience uses to identify themselves, or if the survey needs to be shorter, clearer, etc.

Web Accessibility

Creating an accessible survey also requires intentionally considering the platform you use, the language you use, the format of your questions, and any descriptive content you include. A quarter of the population has a disability of some kind; making your form or survey as accessible as possible will allow all respondents to complete it, whether they have a disability or not. Increasing accessibility means your questions will be understandable to more of your audience, and should increase both quantity and quality of the responses you receive.

To start, review the accessibility features of the digital tools you are using to collect data, and check if they comply with Section 508 and Web Content Accessibility Guidelines (WCAG) standards. We’ve found that most commonly used formbuilder tools (Formstack, FormAssembly, Formsite, Survey Monkey, Gravity Forms, etc.) are generally compliant, but that certain features or formats within each tool may not be accessible.

An incredibly comprehensive summary of accessible survey design from Online Surveys covers both accessibility in content and formatting. Overall, you want to make sure content is clearly written and easy to read, and ensure all users can complete and submit forms, including those using screen readers. Some best practices for written content are to:

• Use simple language and short sentences – aim to write to a 6th grade reading level. The Plain English Campaign website provides free guides that you may find useful.

• Avoid unnecessary jargon, or define it in your survey. If you use acronyms, expand these the first time they are encountered on a page.

• Break up large amounts of text into a bulleted list for readability.

• Provide an indication of how long the survey will take to complete at the beginning of your survey.

• Use italics and bold for emphasis rather than capitalization, font color, or font size. Use emphasis on words rather than entire sentences, and use sparingly.

• Don’t ask leading questions. While you may get more positive responses by asking “How much did you enjoy our fantastic event?”, you won’t get the most accurate responses. Review and refine your questions to make sure they don’t lead to bias in the data you collect.

Structure and Format

Accessibility covers more than just the language you use. One of the trickiest parts of designing an accessible survey or form is ensuring that the structure of the tool and the formatting of questions work together to make it easy to complete (and respond accurately!).

Overall, aim to keep things as short and sweet as possible. Respondents are giving you the precious gift of time and information, and ensuring your process is straightforward and easy to complete is the best way to thank them. Ask only as many questions as you need to, and always be on the lookout for ways to say more with less. People are less likely to finish a long and confusing survey, or may not fully read questions that are overly verbose before answering.

The order and structure of questions are important as well. Keep questions about the same theme or topic together and, when possible, use conditional logic to adjust questions if there are opportunities for shortening the survey for some respondents.

For example, if you have a set of questions about the experience of using your online donation form, you could first ask: “Have you ever donated through our online donation form?”, and use that answer to hide the subsequent questions when they are irrelevant to an individual.

Good structure can minimize the number of questions the average person has to answer, and can help keep respondents focused.

Question Format

A second accessibility layer is the layout of questions and answers within your survey. There are many formats you can use for your questions (radio-buttons, drop-down list, checkboxes, scales, etc.), and not all question formats are equal. A bad layout can be confusing or overwhelming and can bias people into answering questions incorrectly. Here are some high-level guidelines for formatting your survey questions:

• Make sure your answer types match the way people will respond: questions with the potential for multiple answers should be multi-select, or come with clear instructions for how to choose a single response so that people don’t get frustrated trying to answer correctly.

• Always provide an option to not respond: if you require a question, make sure there is a ‘not applicable’ so that people don’t get stuck if there’s not an appropriate selection for them. There are some cases where this may not be necessary, but be extra cautious about making a question required without an N/A option if you aren’t 100% sure that it’s always answerable.

• Use vertical answer layouts rather than horizontal layouts when there are more than a few answer choices: Vertical works for simple yes/no questions but reduces readability when there are more options.

• When asking respondents to rank using a scale, include the scale in the answer option itself: Don’t make your participants have to work to interpret a text-scale just to give you an answer – make sure that it’s easy for people to understand what each selection for an answer means.

• Test and refine numeric scales to make sure they do not bias your responses: Especially when asking about sensitive topics, answer options that emphasize high or low ranges may influence respondents to choose an answer that is artificially low or high in order to ‘fit’ in with the provided options.

Takeaways:

• Use the language and terms that make sense to your target audience.

• Work with your audience to refine the format of surveys and forms so that they can be easily understood and completed.

• Use data collection tools that conform to web accessibility standards.

• Make sure that the content of your forms is accessible by people who may have visual impairments and use screen readers or other adaptive technologies.

• Look at your surveys as a whole and make sure the order, groupings, and format of your questions helps to guide respondents in completing the surveys or forms.

Overall, keep your target community in mind in every step of the design process – as you think about what you want to ask, how you ask for it, and how you display your questions and answer options.

Case Study: Sacred Heart Community Service

Sacred Heart Community Service (SHCS) is a nonprofit organization that helps people in need with food, housing, health services, and legal assistance. Sacred Heart works with about 50,000 constituents, providing access to critical resources and tools for self-sufficiency and working to grow community power and leaders. SHCS use constituent data to continuously improve their programs, advocate on behalf of and build the agency of their constituents, and to report back on their successes to funders.

As a social service organization, Sacred Heart’s members must go through multiple steps and forms to verify if they qualify for various services and assistance programs. Sacred Heart worked with Percolator Consulting to create a unified intake system that eases the burden of this process. Throughout the development process, the Sacred Heart team took care to make the form as streamlined and accessible as possible for their diverse community. They translated the form into four languages most commonly spoken by their constituents, and considered question order to ensure that the intake process was as easy as possible for respondents.

When applying for membership, language selection is the first question that appears, so that people can continue the application process in the language that is most accessible to them. The second set of questions is used to identify if the applicant is already in the system, so they don’t have to fill out any information that SHCS already has.

On the second page of their intake, questions about what exact services an applicant is interested in appear, and then additional contact and demographic information with the option to provide it later if the applicant is short on time. Throughout the entire intake form and the rest of the application process, SHCS has worked hard to collect the right details at the right time so that people don’t have to provide information for services they don’t want or qualify for, as well as ensure that Sacred Heart has the data they need to communicate the impact of their work, and to advocate for better policies and systems that affect their constituents.

Highlighted Resources:

• Inclusive Language Guide | APA

• Creating Accessible Surveys | Online Surveys

• Forms Tutorial | Web Accessibility Initiative (WAI) | W3C

• Get Started with Web Accessibility | Salesforce Trailhead