I bumped into the acid3 test, and I noticed it didn't pass in Chrome anymore. Would anyone have any context as to why this is? It's a little bit odd to me that backwards compatibility was broken in at least 3 ways here.
I don't know because this isn't my area of expertise, but when I use my Firefox browser to load the page with the developer console open, it throws two XML parsing errors.
So I figure it's one of three things going on here. One, modern browser makers just don't bother testing Acid3 anymore, and bugs have crept into the browser. Two, modern browsers knowingly break Acid3 because they prefer to implement performance enhancements that are not strictly compliant with standards. Three, updated standards have added syntax or rules that invalidate some of the strange stuff that is packed into Acid3.
The modern "standard" is what's called "living". Which means that any part of it can change at any time. This is unlike the old standards that were fixed (if not always adhered to).
Browsers aim to be backwards compatible with popular or influential sites, not synthetic tests.
You can click/tap on “Acid3” after the test is done and get some details.
E.g. on my phone (iPhone, up-to-dare, I think) which scored 98/100 I get:
Failed 2 tests.
Test 23 failed: expected '14' but got '5' - wrong exception for createElementNS('null', ':div')
Test 25 failed: wrong exception
Total elapsed time: 0.49s
The first one means it passed invalid parameters to createElementNS and expected a NamespaceError (14) exception but an InvalidCharacterError (5) exception occurred instead.
(I’m waiting at the drs office, and am happy to pass the time looking up error codes.)
ACID has always been about testing obscure edge cases and weird parts of the HTML spec, and sometimes the way that the spec was written in the past differed from "in-the-wild" behavior. These days, in the post-W3C era, the HTML specs try hard to capture behavior as seen in-the-wild. Basically, ACID doesn't test the modern spec.
Acid3 was not updated when people decided to change the specifications. There are 3 things that are known to fail. Two of them are related to one change (see https://github.com/whatwg/dom/issues/319#issuecomment-296345... for details on that change). The third test is one where Acid3 mandates something that every browser did but was wrong per the specification, and it took several years for people to notice (see https://bugzilla.mozilla.org/show_bug.cgi?id=1311329 for some more details).
"Acid3, in particular, contains some controversial tests and no longer reflects the consensus of the Web standards it purports to test, especially when it comes to issues affecting mobile browsers. The tests remain available for historical purposes and for use by browser vendors. It would be inappropriate, however, to use them as part of a certification process, especially for mobile browsers."
http://acid3.acidtests.org/support-a.png is a resource that page attempts to load that 404's now. I'm guessing that's part of the reason it doesn't score 100%. Not the browser's fault.
This is by design.
By the specs, browsers should not display a favicon if the file 404s.
But that file contains a valid favicon, so rendering it is a failure.
[source: Wikipedia]
Bucket 3: DOM2 Views, DOM 2 Style, CSS3 selectors and Media Queries
By April 2017, the updated specifications had diverged from the test such that the latest versions of Google Chrome and Mozilla Firefox no longer pass the test as written. Hickson acknowledges that some aspects of the test were controversial and has written that the test "no longer reflects the consensus of the Web standards it purports to test, especially when it comes to issues affecting mobile browsers."
As I recall, the Firefox of the time didn't pass Acid3, either, and there was a minor controversy over the fact they changed the test in a way that made it friendlier to Firefox:
Initially, Firefox 4 scored 97/100, because it did not support SVG fonts. Later, Firefox 4 scored 100/100, because the SVG font tests were removed from Acid3.
According to Mozilla employee Robert O'Callahan, Firefox did not support SVG fonts because Mozilla considered WOFF a superior alternative to SVG fonts.[29] Another Mozilla engineer, Boris Zbarsky, claimed that the subset of the specification implemented in Webkit and Opera gives no benefits to web authors or users over WOFF, and he asserted that implementing SVG Fonts fully in a web browser is hard because it was "not designed with integration with HTML in mind".[30]
For people who don't follow modern web standards, a modern web browser getting 100/100 is actually failing tests. You should get 97/100, failing tests 23, 25, and 35.
The site itself is no longer maintained and updated to reflect changes to standards.
If you're asking specifically why the test isn't being maintained, I can somewhat speak to that.
The first Acid test was a relatively simple test that could visually demonstrate if browsers implemented CSS. When IE became the dominant browser, Microsoft essentially stopped all development on IE. At the time, IE had a very broken implementation of CSS (although, it should be noted, IE6 was the best implementation of CSS when it came out, as everybody else was just as bad). The Acid2 test was created in large measure specifically to goad IE into improving its standard compliance (mostly on CSS). As a result of the publicity of the Acid2 test, the other browsers fixed their engines to pass Acid2, which put pressure on IE to fix their engine.
Since it had been effective in pushing browsers to implement the CSS2 standard correctly, the author turned to creating Acid3 to push browsers to implement more technologies. Whereas Acid1 and Acid2 were largely about CSS layout, Acid3 focuses more on testing JS API implementations. Acid3 was heavily criticized for letting browsers score well just by having a very thin, minimal surface implementation of the feature instead of testing thorough correctness, as well as for being very opinionated in pushing browsers to implement specific technologies (SVG fonts being the big one).
By the time Acid3 came out, there was already a change in the development of web browsers. Microsoft was once again participating in the standards compliance process and trying to drive their browser to modern compliance, so all of the major browser vendors were very active in discussion about implementation. Also, the W3C changed its specification process to rely more heavily on having a thorough compliance testsuite that had two passing independent implementations before a standard could be considered finalized. The CSS2.1 testsuite wasn't complete at the time these tests were developed, but has now come out, with almost 10,000 individual tests in them, far more thorough than any of the Acid tests could claim to be.
Since the Acid tests no longer had value in being the most comprehensive tests of their underlying functionality, nor in compelling browser developers to be competitive in standards support, their developer stopped working on the project. Indeed, since Acid3 fails at even being a comprehensive test for what it tests, its author publicly stated that he isn't interested in updating it when specifications update.
Do you know what changed regarding 23, 25, and 35, and why it was at the time considered correct to pass them, but now considered correct to fail them?
That's not super helpful, for example 25 just says "wrong exception". The original commenter seemed to know exactly why these 3 specific tests were failing, I'm also curious what specific specs changed that are no longer true.
I gave the links in https://news.ycombinator.com/item?id=18942546 . Essentially, the exception being thrown changed in 23 and 25, and 35 was actually incorrect to begin with, but no one realized it until browsers started implementing Shadow DOM.
Has it really been 10+ years since the "acid" tests debuted? I can remember when these things were the height of the browser wars. I guess I should be glad that we don't really need them anymore.
I bumped into the acid3 test, and I noticed it didn't pass in Chrome anymore. Would anyone have any context as to why this is? It's a little bit odd to me that backwards compatibility was broken in at least 3 ways here.
I don't know because this isn't my area of expertise, but when I use my Firefox browser to load the page with the developer console open, it throws two XML parsing errors.
So I figure it's one of three things going on here. One, modern browser makers just don't bother testing Acid3 anymore, and bugs have crept into the browser. Two, modern browsers knowingly break Acid3 because they prefer to implement performance enhancements that are not strictly compliant with standards. Three, updated standards have added syntax or rules that invalidate some of the strange stuff that is packed into Acid3.
The modern "standard" is what's called "living". Which means that any part of it can change at any time. This is unlike the old standards that were fixed (if not always adhered to).
Browsers aim to be backwards compatible with popular or influential sites, not synthetic tests.
You can click/tap on “Acid3” after the test is done and get some details.
E.g. on my phone (iPhone, up-to-dare, I think) which scored 98/100 I get:
Failed 2 tests. Test 23 failed: expected '14' but got '5' - wrong exception for createElementNS('null', ':div') Test 25 failed: wrong exception Total elapsed time: 0.49s
The first one means it passed invalid parameters to createElementNS and expected a NamespaceError (14) exception but an InvalidCharacterError (5) exception occurred instead.
(I’m waiting at the drs office, and am happy to pass the time looking up error codes.)
ACID has always been about testing obscure edge cases and weird parts of the HTML spec, and sometimes the way that the spec was written in the past differed from "in-the-wild" behavior. These days, in the post-W3C era, the HTML specs try hard to capture behavior as seen in-the-wild. Basically, ACID doesn't test the modern spec.
See, for one example: http://www.howtocreate.co.uk/SGMLComments.html#simple
Acid3 was not updated when people decided to change the specifications. There are 3 things that are known to fail. Two of them are related to one change (see https://github.com/whatwg/dom/issues/319#issuecomment-296345... for details on that change). The third test is one where Acid3 mandates something that every browser did but was wrong per the specification, and it took several years for people to notice (see https://bugzilla.mozilla.org/show_bug.cgi?id=1311329 for some more details).
The Acid3 test on that website is officially unmaintained: https://lists.w3.org/Archives/Public/www-archive/2017Jul/000...
What does it mean by "71 pass"?
The test is scored out of 100, so presumably it means getting a 71+ score
My understanding with these kinds of tests is that the only passing grade is 100/100
I think that’s inadvertant and doesn’t mean anything.
Sorry, it was a typo. I was using Chrome 71 when I bumped back into acid3, and the original title read "Chrome 71".
After checking in other browsers I edited the title but missed the 71. Fixed! TGIF
I think the original title was "Chrome 71 doesn't pass the Acid3 test anymore". Seems like it got edited, but incorrectly.
"Acid3, in particular, contains some controversial tests and no longer reflects the consensus of the Web standards it purports to test, especially when it comes to issues affecting mobile browsers. The tests remain available for historical purposes and for use by browser vendors. It would be inappropriate, however, to use them as part of a certification process, especially for mobile browsers."
http://www.acidtests.org/
Chrome is still 97/100, but honestly that's a pretty old test and things have changed.
Someone remembers XHTML..?
Also getting 97 with Firefox.
Safari is at 98, which I guess now means slightly worse.
94 on Firefox Developer Edition 65.0b12
98/100 in Safari for me.
Acid2 is currently not passing either in Chrome (71.0.3578.98). I'm surprised these never got turned into automated unit tests.
From Wikipedia: "The test fails when browsers become compliant with current CSS collapse and margin standards."
http://acid3.acidtests.org/support-a.png is a resource that page attempts to load that 404's now. I'm guessing that's part of the reason it doesn't score 100%. Not the browser's fault.
It loads fine for me. Looks like this https://imgur.com/a/lrTMBmE
If you wget the file you'll see this:
--2019-01-18 13:46:40-- http://acid3.acidtests.org/support-a.png
Resolving acid3.acidtests.org (acid3.acidtests.org)... 75.119.197.251
Connecting to acid3.acidtests.org (acid3.acidtests.org)|75.119.197.251|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2019-01-18 13:46:40 ERROR 404: Not Found.
This is by design. By the specs, browsers should not display a favicon if the file 404s. But that file contains a valid favicon, so rendering it is a failure. [source: Wikipedia]
The status is 404, but the body contains image data. I assume this is intentional.
Try curling it.
It's actually sending a PNG file, but with a 404 response.
On my machine:
Chromium 71: 97%, blocks 2&3 silver, pauses at 24, 64.
Firefox Quantum 64: 97%, blocks 2&3 silver, pauses at 24, 64, 65.
From Wikipedia:
Bucket 2: DOM2 Core and DOM Events
Bucket 3: DOM2 Views, DOM 2 Style, CSS3 selectors and Media Queries
By April 2017, the updated specifications had diverged from the test such that the latest versions of Google Chrome and Mozilla Firefox no longer pass the test as written. Hickson acknowledges that some aspects of the test were controversial and has written that the test "no longer reflects the consensus of the Web standards it purports to test, especially when it comes to issues affecting mobile browsers."
As I recall, the Firefox of the time didn't pass Acid3, either, and there was a minor controversy over the fact they changed the test in a way that made it friendlier to Firefox:
https://en.wikipedia.org/wiki/Acid3
Latest Chrome: 97% IE 11: 100%
Acid3: FAIL
Fails the back button test.
Internet Explorer 11 .... 100/100 !!
Quirks or strict mode?
Sounds like it is time for Acid4. After all, those test did a great job at showing browser incompatibilities.
There’s html5test.com and css3test.com
The google bot scored 45% on css2test.com.
For people who don't follow modern web standards, a modern web browser getting 100/100 is actually failing tests. You should get 97/100, failing tests 23, 25, and 35.
The site itself is no longer maintained and updated to reflect changes to standards.
thanks - this was helpful.
ha, did not know that.
IE gets 100/100 ... while firefox got 97. That explains it!
"IE beat us, let's change the rules of the game."
It doesn't surprise me, after seeing all the other stuff that happens in the web community...
care to explain why?
If you're asking specifically why the test isn't being maintained, I can somewhat speak to that.
The first Acid test was a relatively simple test that could visually demonstrate if browsers implemented CSS. When IE became the dominant browser, Microsoft essentially stopped all development on IE. At the time, IE had a very broken implementation of CSS (although, it should be noted, IE6 was the best implementation of CSS when it came out, as everybody else was just as bad). The Acid2 test was created in large measure specifically to goad IE into improving its standard compliance (mostly on CSS). As a result of the publicity of the Acid2 test, the other browsers fixed their engines to pass Acid2, which put pressure on IE to fix their engine.
Since it had been effective in pushing browsers to implement the CSS2 standard correctly, the author turned to creating Acid3 to push browsers to implement more technologies. Whereas Acid1 and Acid2 were largely about CSS layout, Acid3 focuses more on testing JS API implementations. Acid3 was heavily criticized for letting browsers score well just by having a very thin, minimal surface implementation of the feature instead of testing thorough correctness, as well as for being very opinionated in pushing browsers to implement specific technologies (SVG fonts being the big one).
By the time Acid3 came out, there was already a change in the development of web browsers. Microsoft was once again participating in the standards compliance process and trying to drive their browser to modern compliance, so all of the major browser vendors were very active in discussion about implementation. Also, the W3C changed its specification process to rely more heavily on having a thorough compliance testsuite that had two passing independent implementations before a standard could be considered finalized. The CSS2.1 testsuite wasn't complete at the time these tests were developed, but has now come out, with almost 10,000 individual tests in them, far more thorough than any of the Acid tests could claim to be.
Since the Acid tests no longer had value in being the most comprehensive tests of their underlying functionality, nor in compelling browser developers to be competitive in standards support, their developer stopped working on the project. Indeed, since Acid3 fails at even being a comprehensive test for what it tests, its author publicly stated that he isn't interested in updating it when specifications update.
Do you know what changed regarding 23, 25, and 35, and why it was at the time considered correct to pass them, but now considered correct to fail them?
You can see details by loading the test page and then clicking on the 'A' of 'Acid3'.
That's not super helpful, for example 25 just says "wrong exception". The original commenter seemed to know exactly why these 3 specific tests were failing, I'm also curious what specific specs changed that are no longer true.
I gave the links in https://news.ycombinator.com/item?id=18942546 . Essentially, the exception being thrown changed in 23 and 25, and 35 was actually incorrect to begin with, but no one realized it until browsers started implementing Shadow DOM.
Current Safari Mobile gets 98/100
Microsoft Edge gets 100/100
Safari Version 12.0.2 (14606.3.4) 98/100
66/100 on Firefox and after a pause it shoots up to 96/100! 97/100 on Brave.
Has it really been 10+ years since the "acid" tests debuted? I can remember when these things were the height of the browser wars. I guess I should be glad that we don't really need them anymore.