Key Findings of the Research

Survey 1: Perceptions

Organisations were surveyed to determine their response to digital accessibility requirements and to attempt to gauge their progress in embedding accessibility into an organisational Digital Accessibility Maturity Model (DAMM), so that it becomes standard operating procedure.

The organisational survey found that very few are using any type of maturity model for accessibility.

Using a DAMM is seen as a method for determining an organisation’s high-level approach to accessibility, measuring their current achievement and establishing goals for the future. This typically involves taking a baseline measurement (normally an audit) and then setting goals and establishing a method and timeline for regular testing as they strive to achieve those goals.

While most organisational respondents (90.48%) stated that digital accessibility was important to their organisation (Figure 4), only 72.09% had their website evaluated for accessibility (Figure 5).

Figure 4: Is digital accessibility importance to your organisation?

Figure 5: Has your website ever been evaluated for accessibility?

 

Of concern, is that 47.62% of the evaluations were completed internally as opposed to being evaluated by external validation (21.43%) (Figure 6).

Figure 6: If the website was evaluated, who performed the evaluation?

Most of the organisations who responded that an evaluation had been conducted, stated they evaluated for both technical compliance and for usability by people with disabilities (Figure 7).

 

Figure 7: What was the website evaluated for?

 

The evaluation of digital material was even less for mobile (58.14%) and only 41.19% for applications (Figure 8 and 9).

Figure 8: Has your mobile website been evaluated for accessibility?

Figure 9: Have your applications been evaluated for accessibility?

Only 9.3% of organisational respondents stated they carried any level of certification (Figure 9).

Figure 10: Does your website have any type of certification?

 

This would seem to be at odds with the responses to the question regarding the importance of accessibility compliance.  Responses indicate no organisations felt compliance was either ‘not useful’, ‘unnecessary’ or ‘a waste of time’ (Figure 11).

Figure 11: Do you feel accessibility compliance is...?

 

Most respondents stated that digital accessibility was ‘a standard operating procedure’ with very few stating it was ‘not a consideration’ or ‘not defined’ (Figure 12).

Figure 12: Digital accessibility in our organisation is...

 

The survey asked respondents if they felt their website was accessible, and 55.81% felt it was accessible (Figure 13), with 23.26% stating they meet the Web Content Accessibility Guidelines (WCAG), Version 2.0 to Level AA (the current Australian requirement), and 23.26% stating they met WCAG 2.1 to Level AA (the new version of the standard adopted in many countries. The Digital Transformation Agency (Australian Federal Government) recommends complying with 2.1 of WCAG, although this is not yet being formally required (Figure 14).

Figure 13: Do you think your website is accessible?

 

Figure 14: To what standard do you believe it is accessible?

It is encouraging that 67% of respondents still plan to do more evaluation, and understandable that 16.28% are unsure (Figure 15). However, it is discouraging that 16.28% state they are not planning to do more evaluation. Perhaps the question could have been phrased as "do you want to have more websites or applications evaluated" as it might not have been phrased in a way to capture those who would like to do more. Perhaps some respondents were discouraged by their organisations not being willing to do more evaluation and remediation. This is another question that might bear more investigation.

Figure 15: Do you plan to have your website or applications evaluated?

 

Organisations were asked whether they used an accessibility standard for procurement when tendering for new ICT projects.  This would refer to standards such as EN 301549 in Europe and AS EN 301549 in Australia (Figure 16).

Figure 16: Use of accessibility standard for procurement when tendering

 

The organisations were asked what type of validation they required to assess the tender compliance with standards. It is clearly stated in standards such as EN 301549, that the organisation tendering for new ICT products and services should state what type of proof was required.  This can vary from a signed statement of compliance, to examples of websites they believe are accessible, to third-party validation from accessibility experts (Figure 17).

 

Figure 17: Accessibility validation required for assessing tender responses

 

When organisations were asked whether they provided training to staff, 67.44% stated that it was provided (Figure 18).  Respondents were asked for the most important digital accessibility needs in their organisation and 19% of respondents stated training is still needed.  Equal numbers stated that resources to fix digital material and internal support were the most critical needs (21.4% for each) (Figure 18).

Figure 18: Digital Accessibility Training

It would seem from the above results, that while organisations understand that digital accessibility is important, they do not grasp the importance of establishing a method for determining compliance and setting goals for improvement. Most respondents felt their website was accessible (Figure 13), and this would seem to be at odds to the answers from individual users in the second survey, which is described in the next section.

Key Points:

  • The majority of organisations (90.48%) state that digital accessibility is important to their organisation.
  • However, only 72.09% had undertaken an evaluation of the website and even less for mobile and applications.
  • The majority of evaluations are completed ‘in-house’, with only 21.43% using external accessibility professionals.
  • Of those organisations undertaking evaluation, 52.38% evaluated for both technical and user testing by people with disabilities.
  • Most organisations do not carry any level of certification (77%).
  • All organisations stated that accessibility compliance was either necessary (51%), a good investment (33%) or useful (16%) with no negative responses.
  • 44% of organisations stated that digital accessibility is standard operating procedure, 35% that it was sometimes considered (35%) or incorporated depending on budget (14%), with 5% stating it was not defined, and only 2% that it was not a consideration.

Survey 2 – Expectations

With such great data from around the world it is interesting that over 80% of users believe that the current accessibility laws in their country are not sufficient.

Because Australia is Web Key IT’s home, we asked whether respondents understood the Australian complaint procedure and legal digital requirements, including the Disability Discrimination Act 1992 which is administered by the Australian Human Rights Commission. It would appear people from outside Australia are also familiar with the Australian legal situation (Figure 19). W3C Web Accessibility Initiative provides detail of international standards on their website, W3C WAI International.  

Figure 19: Understanding of Australian procedures

 

In order to understand the responses and demographic, we asked participants if they considered themselves as having a disability that would affect their education, mobility or employment, these are the same parameters used by census in Australia which gives the often-cited number of almost 1 in 5 or 19.8% of the population. In Figure 20, we can see that the split was very different for the respondents to this survey, with the breakdown showing approximately 55.42% having a disability and 44.58% not having a disability. Perhaps this could be due to the fact that people with a disability were either more interested in participating in the research or that the groups the researchers sent the survey to had a greater representation of people with disability. 

 

Figure 20: Respondents identifying themselves as having a disability

This breakdown will also play a part in the question regarding the most problematic issues commonly found on websites.  While 55% of respondents replied that they had a disability, 5% of those do not associate with a disability support group.  It is commonly assumed people with disability have help and training to know how to make things easier for their digital use through the settings on their operating systems. Aside from that being a fairly bold assumption, here we have evidence that not everyone is tapping into that resource and that is assuming these groups all have the appropriate resources and information in the first place.

 

Respondents were asked whether there were other issues affecting their ability to use digital products. As can be seen in Figure 21, financial limitations (15.58%), age-related issues (13%) access to technical assistance (7.79%) and lack of computer skills (3.9%) were mentioned.  Financial limitations may be related to the cost of assistive technology and computer equipment to suit a person’s abilities. Another factor is likely the disability pay gap – a documented difference between what non-disabled and disabled workers earn.  Add to that, the potential need for more expenses such as medication, specialists, and assistive technologies. There were also responses for difficulty with access to technical support and advice. Age-related issues may relate to vision limitations, lack of confidence using computer equipment, or even shaking hands among many possibilities.

  

Figure 21: Other issues that affect use of digital products

 

We asked the respondents to identify the most common accessibility issues they faced with using a website.  The top 5 are shown in Figure 22 while 23 displays the total list of issues faced by users.

  • Poor Design
  • CAPTCHA or other robot detection measures
  • Poorly designed forms
  • Difficulty finding information
  • Website structure – including headings, menus etc.

 

Figure 22: Top 5 accessibility issues faced on websites

In Figure 22 and 23, it should be noted that we allowed participants to select as many items as they felt appropriate.  We also allowed for people to nominate issues under ‘other’. It is clear from these results that most of these issues would affect people with or without disability.  Many of us can relate to a user being frustrated because the website is designed in such a manner that they cannot find information or navigate to areas of interest.  As other research has discovered, the use of CAPTCHA and other robot-detection measures continues to be a great cause of frustration for users.  For many users, such as those using keyboard-only or screen-reader technology, they are unable to complete a task if such a measure is used.  While newer technology such as Re-CAPTCHA makes that a little easier, it is still unusable for a great many users.

 

Issues Facing Users - Key Points:

  • Issues that most affect users with disability also affect a great many more people. We can all relate to the frustration we feel when using poorly designed websites.
  • By creating an accessible website, you create a website that allows access to a far greater percentage of the user population.

 

Figure 23: All issues identified by users as causing difficulty using websites

 

When considering the business case for digital accessibility and the issues shown in Figures 22 and 23 above, it is not surprising that 65.85% of respondents stated they have left a website because of its inaccessible design.  Given that 55% of the respondents stated they had a disability, that clearly demonstrates that some of the users without a disability had also left websites because of poor design and inaccessible features.

 

Chances are that if you had a company website, you would not be happy with 66% of users leaving a website because of accessibility issues (Figure 24).

 

Figure 24: Respondents who have left a website because of its accessibility issues

 

In Figure 25, we can see that 59.26% of the users had complained to the owner of a website about its accessibility and advised the owner of issues that made their website difficult or impossible to use.

Figure 25: Respondents who advised a website owner of accessibility issues

Many assume that people who cannot use a particular website will simply find a different website to gain the required information without lodging a complaint except for cases where there is no alternative, such as a government website.  However, in Figure 25, we see that more people had complained than not.  However, this again may be due to the people who were interested in the issue of digital accessibility. 

We asked respondents who had complained, which method they had used to make their complaint known, with the vast majority using email or an online form. This would indicate that having an email address available or an online form for complaints is a valid method.  Very few (less than 10%) used a telephone to advise the website owner of the issue.

Following the questions about complaints of the website’s accessibility, we asked respondents if they had gone back to check if the issue had been fixed.  While almost 60% stated they had, it is a concern that over 40% did not go back to check.  It is likely that these individuals did not go back to the website at all after encountering an issue that affected their ability to use the website.  This would mean that the organisation has probably lost this user – something few businesses could afford to do.

Of those users who did go back and check, approximately 12% found the issue had been fixed, while almost 40% stated that it had not been fixed.  Some users stated that they were not sure if it had been fixed or not (5%).

Most people make a first impression of an organisation from their advertising material, public profile, or even the look of the organisation’s premises.  The same is true of an impression gained of an organisation by the accessibility of the website.  (Figure 26)

 

Figure 26: Does a website's design or accessibility affect your impression of the organisation?

As shown in Figure 26, almost all respondents stated that their impression of the organisation was affected by their website’s design or accessibility.  Most people can probably relate to this when using a particularly cumbersome or poorly designed website. We assume this to be a lack of ability of the organisation, or worse a lack of care about the user’s impression or whether they could use the website for its intended purpose.

 

To further clarify this issue, we asked that if there was a choice of website for the information or service required, would the website’s accessibility or design motivate the individual to choose a different website (Figure 27).

Figure 27: Would the respondent look for a different website if there was a choice

As can be seen in Figure 27, almost no one would return to a website that had issues affecting its usability.  This agrees strongly with the responses shown in Figure 26 which demonstrates how individuals view an organisation with an inaccessible website.

Users seem equally frustrated with the lack of enforceable legislation in their country regarding digital accessibility (Figure 28).

Figure 28: Should digital accessibility be better legislated

In Figure 28, we see almost identical responses to the question as to whether they would choose a different website if possible, and those who felt more enforceable legislation was required.

Key Points:

  • most users form an impression of an organisation by its website;
  • users would switch to a different organisation for information if it was available elsewhere;
  • users want legislation improved;
  • users who complained, often did not go back to the website to check if the issue has been fixed, meaning they may not go back at all and resulted in a lost customer;
  • the same issues of design, robot-detection methods, poor form design, difficulty finding information and structure continue to top the polls for most problematic issues for users;
  • 60% state that issues raised in evaluations were fixed, with only 12% stating this had not happened.
  • over 80% of users would switch to a different website when faced with accessibility issues, if there is a choice and
  • If there isn’t a choice of website, the user is faced with the issue of either contacting the organisation for another method of receiving the information, or requesting outside assistance to help them use the website.

 

Discussion – The Reality

The results of the surveys discussed above show a clear discrepancy between what organisations believe about their adoption of digital accessibility guidelines displays and the experience of their users.

For example, in Figure 29, we see that 79% of organisations believe that digital accessibility in their organisation is either ‘standard operating procedure’ or ‘sometimes considered but not always’.

 

Figure 29: Digital accessibility in our organisation is...

Yet, users do not agree, with 65% of users stating they had left a website because of its poor accessibility and design (Figure 24) and the same accessibility issues continuing to be cited (Figure 30).  These issues have consistently been identified in research and little or no change has occurred.

Figure 30: Most common accessibility complaints

Users (95%) clearly stated the design or accessibility of the website affected their impression of the organisation, in much the same way that the physical appearance of a business affects our impression of the organisation, their values, their services etc. (Figure 25).  Users strongly also stated strongly that if there was a choice of website for the information they required, a website’s accessibility or design would motivate them to choose a different website (81.7% in the affirmative, 3.7% negative, 14.63% unsure) (Figure 27).

It is encouraging to see that 67.44% of organisational respondents still plan to have their website or applications evaluated further, however it is not apparent if the plans are for internal or external evaluation (Figure 6). The discrepancy between results from internal and external evaluation may answer the question regarding the disparity of the organisation’s perception of their accessibility and the experience from the user.  Internal validation could be similarly to composing, answering and marking your own test, with little credence given to self-assessment because of the obvious bias or lack of understanding of requirements. This is also dealt with in the assessment of the Australian National Transition Strategy for Website Accessibility (NTA). If credence is given to the answers from individual organisations covered in the strategy, it would seem that most organisations were compliant at the end of the NTA, however the results of Conway’s analysis states the opposite to be the result with few obtaining compliance when tested. (Conway 2014)

While almost all organisations state accessibility is important to their organisation, (Figure 12), they were not all involved with assessing the accessibility of their websites, and few organisations are making digital accessibility a core aspect of their business (Figure 13).  Indeed, of those organisations who had their website, application or mobile website assessed, approximately 60% stated they had fixed the issues highlighted. (Figure 31).

Figure 31: For organisations, was remediation completed and website re-tested?

 

From the user perspective, almost 60% went back to see if the issue (Figure 32), had been fixed. Of those who did go back and check, only 12% found that the issue had been fixed satisfactorily (Figure 33).

Figure 32: User checked to see if issue has been fixed

 

Figure 33: For the user, was the issue fixed satisfactorily?

Key Points:

  • Organisations largely feel their digital products are accessible, but users do not agree.
  • Users form their opinion of an organisation through the accessibility or design of the website in the same manner they view a physical building or the attitude of staff in the organisation.
  • Users will leave a website if:
    • it is not easy to use;
    • is not accessible and usable, and
    • if they have an alternative website choice.
  • If there is no choice for an alternative website, the user is faced with the issue of getting help from the organisation (telephone or email or in person), or having to surrender their security/privacy/independence to ask someone to assist them.

Take me to the next section: Summary