Evaluating Your Process and Outcomes
Regular assessment is necessary to ensure that your web accessibility plan is working and on track. You must take time to regularly reflect on the effectiveness of your organization’s process (implementation plan) and its outcomes (your website).
Evaluate Your Process
Big changes often take a long time, sometimes years for large organizations. Due to lengthy timelines, periodic assessments of your process are vital. By this, we mean evaluating the efforts made to achieve the goals and objectives (or milestones and activities).
Here is a six-step plan for assessing your process:
- Within your written plan, outline a timeframe to review accomplishments and challenges in your intended process (e.g., every six months).
- Create a review document that captures the work you intend to complete by the end of the timeframe.
- Determine if you require any independent sources of data to be gathered to verify your progress. (Note—this will not be necessary for every goal or activity that is reviewed.)
- Engage in the review. This becomes your formal process assessment.
- Document the assessment. This documentation can take the form of an email or message, or it may be a formal report.
Related Resource
Most evaluations of this kind are not distributed publicly, so it is difficult to find real-world examples. Here is a brief report from Western Illinois University that outlines their progress towards desired accessibility outcomes.
Evaluate Your Outcomes
Ultimately, you determine the effectiveness of your process is by evaluating your outcomes (i.e., an accessible website). In an earlier section, we outlined tools and techniques for evaluating web accessibility. This section outlines how you can organize your evaluation and share your findings with others.
There are seven components of a successful evaluation:
- Purpose—This will influence every other aspect of the evaluation. A baseline to determine where to start your efforts will be broad in scope, versus a focused evaluation to improve a single area.
- Scope—Determine which areas within the site will be included (e.g., subdomains, mobile versions) or excluded (e.g., third-party or legacy content), or if there is a specific area that will be the focus of this evaluation. Establish a technical standard (e.g., WCAG 2.1 A/AA). Decide which assistive technologies will be used in the evaluation.
- Sample—Explore the site and identify common or important pages or templates and other pages that should be included in the evaluation. Then choose the pages and files that represent this sample.
- Standard—Your policy typically determines the standard you will use in an evaluation, but it is a good idea to look for other issues as well. For example, you may be required to meet WCAG 2.0, but you choose to test to WCAG 2.1. because you know it will result in a better end-user experience.
- Format—Although not every evaluation requires a formal report, you should still determine how you will organize your insights, so they are useful to your audience. Most formal reports should include the following:
- Introduction including dates, scope, methods, etc.
- Executive summary with recommendations for next steps. This may be the only part of the report that many people will read.
- Statement of conformance. How well do you meet your accessibility standard? The W3C has outlined WCAG 2 conformance requirements.
- Common issues. Often, the same issue occurs throughout a site or area. Grouping these issues together often makes it easier to make the changes that have the most impact.
- Page-by-page issues. This section is primarily for those who will be making the repairs.
- Evaluation—Review selected pages to determine how well they meet the established technical accessibility standard. Additional issues that do not fall within the technical standard should be noted. If there is no one in your group with this expertise to conduct the evaluation, there are groups and companies that offer these services.
- Plan of Action—Help people decide how to act on the report. Issues that are part of templates or reusable objects should usually be addressed first because a single fix will benefit multiple pages. Issues should also be prioritized based on a combination of severity, end-user impact, ease of repair, and visibility.
Related Resource
- Evaluating your Institution's Web Accessibility—This resource provides additional details on the steps outlined above.
- Website Accessibility Conformance Evaluation Methodology (WCAG-EM)—WCAG-EM provides a good framework for evaluation. The WCAG-EM steps are listed, along with added descriptions for clarification.
- Prioritizing Remediation of Web Accessibility Issues—This brief article lays out some factors to consider to help determine what issues to address first.
