Skip to content.

Sections

You are here: Home » faq
 
1.Getting to know EIAO
2.Question about the evaluations performed by EIAO
3.Developers' FAQ
4.About the UWEM
5.Using and co-operating with EIAO

1.Getting to know EIAO

Why EIAO?

Access for all to the Information Society is a key goal for the European Commission link to EC . Many people, especially people with a disability, meet barriers when accessing the web. Benchmarking can locate potential barriers and fuel the development towards Internet access for all. The EIAO project is developing large scale accessibility benchmarking software. By providing frequently updated data on web accessibility evaluations and deviations from standards of European public web sites, the EIAO will:

Provide quantitative background material for policy-making and targeted actions to improve web accessibility. This may be of special importance for the ongoing discussions about legislation (anti-discrimination laws)for equal rights related to web accessibility.

Promote e-accessibility by raising awareness and encourage competition through benchmarking of web accessibility and guidelines conformance.

Provide input for the discussion on standards and the development of new standards.

Contribute to the Lisbon strategy launched by the European Commission which is striving to make Europe the world's most competitive knowledge based economy within 2010.

What is EIAO?

The European Internet Accessibility Observatory project is developing a prototype for large scale automatic web accessibility evaluation. Evaluation reports will be made available online via a web interface. Reports will give an overview of the accessibility of public web sites across Europe and will provide updates on changes observed in accessibility over time. The Observatory aims to provide monthly reports on the accessibility of around 10,000 European web sites.
EIAO also aims to support the development of e-participation and of e-government solutions by benchmarking accessibility, presenting the benchmarking results online and provide general knowledge about web accessibility. The project is co-funded by the European Commission DG Information Society and Media, under the contract IST004526.

Who is working in EIAO?

EIAO is a project carried out by consortium of European Universities and SMEs (Small and Medium size Enterprises) representing 6 countries: Norway, Denmark, United Kingdom, Italy, Germany and Poland. The project is co-ordinated by University of Agder, Norway.

When will the observatory be in operation?

A test version is planned be in operation and open to the public by early 2008.

How many web sites are intended to be evaluated?

The final version of the prototype Observatory is expected to be capable of processing measurements from around 10,000 European web sites per month.

How can you monitor accessibility across Europe given the different national laws asking for different technical compliance?

We are aware that many countries have developed their national laws and that even if they are based on the same guidelines( WCAG 1.0 / WAI),the technical requirements can differ a lot. This also causes great difficulties in comparing objectively the national accessibility reports. To be able to produce benchmarking results across Europe the monitoring has to be based on the same set of requirements. For this reason we've been asked by the European Commission to develop an European methodology based on WCAG/WAI guidelines. Our project was thus inserted in a Cluster of IST projects working on similar issues. The WAB Cluster is working to develop UWEM the Unified Web Evaluation Methodology which is the common basis of 3 European projects involved in the Cluster. UWEM will allow EIAO to have European results based on a common layer and compare the results in a more objective way. UWEM is developed under the W3C supervision and it has been developed considering the coming WCAG 2.0 version, thus UWEM is a flexible methodology which will be able to adapt to the new coming recommendations.

Will EIAO be an alternative to other evaluation tools like WebExact,Cynthia Say etc?

The Observatory performs individual tests of web sites, but it is not an alternative to the other automatic evaluation tools. The main quality of the Observatory is the ability to aggregate individual tests to large scale benchmarking and to evaluate the development of the results from one period to the next. This is based on the evaluation and sampling procedures described in UWEM . Those are developed to make the evaluation results comparable between web sites and groups of web sites, and also to assure that aggregated results from evaluations at different times are comparable.
Therefore, the role of EIAO will not be to work as an alternative to other automatic checks. It will rather be to identify (among a large number of web sites) the candidates for improvements. The Observatory will only check the web site, retrofitting activities and corrections are responsibility of the web site owner.

For whom the EIAO result be useful?

The Observatory is developed to support e-participation and development of inclusive e-government solutions by providing large scale benchmarking. The results will be useful for anybody who is involved in making decisions related to web accessibility, like policy makers, strategists, public officials, senior managers and anybody interested and operating in e-inclusion field. There is also information available on more detailed level for web developers.

What do you plan to measure beyond accessibility?

The prototype to be finalised within the agreed project contract will only cover accessibility. The tools that are developed can however also be used to collect other information about the web sites, like content type statistics.

2.Question about the evaluations performed by EIAO

What WCAG levels are covered by the Observatory?

The EIAO implementation is based on the Unified Web Evaluation Methodology (UWEM) .level AA (corresponding to priorities 1 and 2).
The Observatory performs only automated testing and therefore cannot cover all tests for a WCAG level. About 2/3 of the WCAG checkpoints can be partly evaluated automatically by the Observatory. These checkpoints are listed in Table 1 in EIAO help (link).

Which web sites are evaluated?

Currently the observatory is using the URL list on which the EC survey of Online Public Services is based on, The user Challenge Benchmarking The Supply Of Online Public Services, 7th measurement. This report has been updated in 2007 and it is available at:http://www.epractice.eu/document/3929 .

How is a web site evaluated?

The web site is scanned with a multithreaded breadth-first crawling approach for URLs. At most 6000 pages are currently downloaded from each site. Subsequently, a random uniform sample of web pages are selected for accessibility evaluation. All automatable tests from UWEM are applied to the web pages. From the results the UWEM accessibility score is calculated and presented on the EIAO web site.

How are the results displayed?

The aggregated UWEM. Scores are presented on the EIAO Web site. The tool can present different reports. A user can choose to see aggregated results for a specific web site, for geographical regions, for business sectors and can get an overview of the accessibility changes within a web site over time.

Why the crawler doesn't evaluate all the pages of a web site?

the crawler downloads 6000 pages within each web site, or the complete web site if less than 6000 pages are available. However, only a random uniform sample of the web pages found are part of the evaluation. Based on the downloaded pages, we can draw representative samples of web pages to be evaluated. For most web sites, the score is calculated from a set of pages smaller than the entire web site.
For details on how we sample from each web site seeUWEM.

What formats is the crawler downloading?

UWEM accessibility evaluation covers (X)HTML and CSS. Other formats, which are no W3C recommendations such as Macromedia Flash are not downloaded and are not part of the evaluation.
For more information on the technologies covered by the evaluation, seeUWEM.

How long will it take for the crawler to reflect in the score the web site changes and improvements recently added?

The Observatory evaluates a web site about once a month, so all the new features and improvments made to a specific web site will be reflected in the next evaluation report. Note that since in most cases only a subset of the site is evaluated, we cannot guarantee smalles changes will be reflected.

Is there any email or contact detail for comment regarding the web sites categorizations?

All sites are categorised into a UNTS and a NACE category. If anyone finds that the categorisation of a web site is wrong, we'll greately appreciate any notice about it. Corrections and comments can be sent to:
feedback@eiao.net
If you click on this email address, it will open a new email on your email application.

Some web sites have been attributed some very bad scores, although they do not seem so unaccessible sites. How can this happen?

There might be several reasons why a web site gets a bad score:

  • The EIAO web accessibility metrics address WCAG 1.0 Level AA,i.e.,there are tests of priority 1 and 2. thus if a site conforms only the WCAG 1.0 Level A the score it gets is lower than if it would have conformed to AA.
  • The evaluation covers a subset of pages from a web site . Even though the Observatory tries to minimise the sample error there might be some deviating results. However, it is highly unlikely that this problem would occur for several evaluations in a row.
  • EIAO covers only the automatable tests from UWEM. Nevertheless, web site owners should take into account all accessibility requirements regardless of whether they can be checked automatically or not and provide a manual check to verify also all the lelements which are not detected automatically by the Observatory.
  • The HTML code of a web pages is not correct. If a page is not valid HTML this might lead to problems for the tool assessing the page. Web site developers and owners should keep in mind the assitive technology also relies on software to parse the web page and thus will most likely encouter similar problems as the EIAO software and hinder or impede the access of people using assistive technology.
  • To have more information on the reports or ask for any specific service or advice, feel free to write to:
    feedback@eiao.net
    If you click on this email address, it will open a new email on your email application.

How can be possible for a web site to be perfectly compliant with a national law and being and having had an unfair score by the Observatory?

National technical requirements may differ a lot. We try to give the most precise and updated results according to the current international recommendations. We suggest web site owners and developers to check the site and, in respect of the national law, take into consideration also the international standards and improve the work in that direction.

Is it possible to see the crawler implementation to get a better understanding of how it works, and possibly suggest improvements?

Yes. Anyone can download all the source code from: http://svn.eiao.net/robacc and any suggestion will be more than welcome!
An install guide is available at http://svn.eiao.net/robacc/INSTALL .
If you have any questions, suggestions or comments, feel free write to:
feedback@eiao.net
If you click on this email address, it will open a new email on your email application.

What are the most common barriers?

According to the crawler latest analyses, the most frequent violation is the use of obsolete features, known as deprecated elements and attributes. For example, using the HTML font element rather than Cascading Style Sheet to control the presentation of the text on a page.
To see the list of updated W3C,recommendations go to http://www.w3.org/TR/ .

3.Developers' FAQ

What is the software architecture of EIAO?

The European Internet Accessibility Observatory consists of:
  • A set of Web Accessibility Metrics (WAMs) compliant with the UWEM automatic test set, based on WCAG 1.0 . Relaxed is used as a basis for our evaluating the WCAG 1.0 checkpoints.
  • An Internet crawler for automatic and frequent collection of web pages for accessibility assessment. For this purpose, the Open Source web crawler HarvestMan has been integrated in the Observatory.
  • A data warehouse with a graphical user interface providing on-line access to collected accessibility data.

What is the sampling strategy?

The sampling strategy is defined and in Chapter 4 of UWEM.

Which Web Accessibility Metrics are used?

EIAO uses the UWEM automatable set of Web Accessibility Metrics( LINK to WAMs ), based on WCAG 1.0 level A and AA checkpoints for evaluating the accessibility of web sites.

What are the metrics and how you can say that an automatic assessment will provide efficient results?

We are aware that accessibility is much more than automatic assessment, and that even including the remaining WCAG checkpoints by a manual test may not identify all the accessibility problems a user might have. But at the same time there is the need to have easy and practical data of accessibility development across Europe. This accessibility data can only be achieved by the use of automatic evaluation, based on a common set of metrics defined according to the international standards in this field. We have defined our metrics starting from interviews done both to stake-holders and disabled people, collecting their needs and shaping the crawler on the answers we have received [For detailed information about our research activities on metrics, user needs' and development, please refer to the EIAO website .Thus, our aim is to have a practical and trusted crawler for massive automatic assessment.
For a clearer picture of a single web site's accessibility each web site developer or evaluator will need to perform manual testing.

How is the score calculated?

To calculate the score for a single web page, the barrier indicator ratio (number of failed tests divided by total number of tests applied) is used. To evaluate and score an entire web site, the Observatory will calculate the average of all web page results. For more information and details, see the related publication on the EIAO website .

How have been selected the websites crawled by the Observatory?

The Observatory has collected representative list of European websites which can provide an indication of the status of web accessibility in Europe. If a site is there it means that it has been selected by one of the sources we have consulted.
If you have any doubt, concern or would like your website not to be listed there anymore, feel free to get in touch with us:
feedback@eiao.net
If you click on this email address, it will open a new email on your email application.

Is it possible to prevent the Observatory from crawling a web site?

Robots.txt is a standard document that can tell EIAO crawlers not to download some or all information from a web server. The format of the robots.txt file is specified in the Robot Exclusion Standard [ROBOTSTXT].

Could you crawl web sites a bit less aggressively?

If a web site supports http if-modified-since or last-modified, the crawler will only download web pages that have been updated since the last crawl. If a site does not support these features, it is impossible for the crawler to know which pages have been changed since the last evaluation. Without such information, the crawler will need to download 6000 pages, or complete web sites, in order to do a representative evaluation of the site.
In other words, by supporting last-modified site owners will drastically reduce the load on a web site. For more info on how to use last-modified see HTTP1.1 .
If you still think we are crawling too aggressively or the crawler is causing you any sort of inconvenience, please provide us some evidences, like weblogs, and comments so that we can improve our work:
feedback@eiao.net
If you click on this email address, it will open a new email on your email application.

What is the software license of EIAO?

All software developed in the project will be released under the GPL License .For software used to carry out evaluations transparency is an essential requirement. The software is free to download, use, review and even to change. This freedom is particularly important for software used for evaluation so that those who are interested can find out how the evaluation really works. It is also possible to propose improvements. The Open Source license ensures access to source code and seems very natural for software developed to evaluate accessibility:
  • Access to code for verification of evaluation method. A fully transparent evaluation method is important for the credibility of the result.
  • Freedom to improve and use the validation methods. Further installations of the system will enable additional teams to experiment and to contribute.
  • Localisation (web interface, development guidelines). Local language is important for accessibility and for acceptance in additional countries.
  • To allow for integration of vendor owned evaluation modules a web service interface is also provided.

4.About the UWEM

What is UWEM?

UWEM means "Unified Web Evaluation Methodology" and it is an European methodology for conformance evaluation and large scale benchmarking. It is developed by the WabCluster, the Web Accessibility Benchmarking Cluster which is a cluster of European projects established to develop a harmonized European methodology for evaluation and benchmarking of web site accessibility. Read more on http://www.wabcluster.org/index.html.

Why there is the need for UWEM when there are WCAGs ?

Even with the same set of guidelines, checking a web site’s accessibility can be carried out in different ways. Therefore there has been a need to develop the Unified Web Evaluation Methodology( UWEM )to ensure that evaluations are compatible and coherent with the W3C's Web Content Accessibility Guidelines ( WCAG 1.0 )part of the broad-reaching Web Accessibility Initiative( WAI ).

Is UWEM an official EU methodology?

The methodology is developed by three European research (FP6 IST) projects in the area of e-inclusion. The projects are grouped under the European Web Accessibility Benchmarking Cluster ( WAB Cluster ). They aim to establish UWEM as the basis for web accessibility evaluation and certification in Europe.
The UWEM development is financially supported by the European Commission DG Information Society and Media .

What are the future plans for UWEM ?

UWEM 1.0 was published in November 2006. Since then the work on UWEM has continued. Several reviews and feedback from accessibility experts has lead to the new and current version UWEM 1.2, with in creased usability and coherence. UWEM are open to comments and contributes: feel free to send comments and any relevant input which can improve the quality of our work.
feedback@eiao.net
If you click on this email address, it will open a new email on your email application.

5.Using and co-operating with EIAO

Is it possible to ask the Observatory for specific checks?

This is currently not included in the EIAO functionality. The EIAO demonstrator currently is developed to observe the accessibility of a given set of public web sites .

Is it possible to use and cite the Observatory's data in a report?

Yes, just make sure to include the reference to the source.

Where can I learn more about web accessibility?

WAI/W3C link and see EIAO related publications.

Can I collaborate with EIAO?

We welcome any inputs and ideas for further development or for improving our work. At the moment we are not hiring, thus if you feel you are interested in the project, you can start sending your suggestions and inputs as a volunteer. To get in touch with us, write us!
feedback@eiao.net
If you click on this email address, it will open a new email on your email application.

I have found the Project Handbook on the EIAO web site, is it possible to use it?

Yes, and in case you want to amend or improve it, please note the Creative Commons Share Alike license conditions .
The Project Handbook is available here .

I represent a company developing a similar tool, can we share information or join the project?

EIAO is an open source project so you have access to the source code. The project is having a six month extension and it will end in February 2008. At this stage we are not planning to take on any new partner. However, you are welcome to contact us.
feedback@eiao.net
If you click on this email address, it will open a new email on your email application.

about new collaboration opportunities such as partnership for an FP7 proposal.

Can I have the reports in other formats?

Currently reports are available in HTML format. We are working to produce reports also in CSV format.

Will the evaluation results be made publicly available?

Yes, once the Observatory will be fully working you can find a link to the

Can we present a subset of your data on another web site, e.g. via a web service?

Yes, this is one of the services we plan to provide. For more information, feel free to write us.
feedback@eiao.net
If you click on this email address, it will open a new email on your email application.

Will the final EIAO software be available to purchase? If so, how much will it cost?

The software is free and can be downloaded when it is ready.To get more information on the licence and instruction, please go here: http://svn.eiao.net/robacc/INSTALL.
Created by yang
Last modified 2008-02-07 09:25