I wish spec editors would stop making spec changes without updating the spec's web platform tests in the process...
1
4
25
Even if you add a test if you do not also have an implementation you might not realize other tests are impacted. And even if you suspect they are it can be rather hard to find them all.
2
10
Sure, but in this case the spec change was accompanied by absolutely no tests... Solving this problem perfectly is hard, I agree. Doing at least something so people will notice the spec change and have some idea of whether they have changed their code to match is not that hard.
1
1
I see, if W3C I believe @plhw3org is gradually making progress on that front, but not sure where it’s at organization-wide. Maybe this is also something @davidbaron should push on during charter review.
2
1
Replying to @annevk @MarcosC @plhw3org
We also need to do something on the Mozilla side where we notice new/changed tests that fail, in areas where we would expect tests to pass.

Dec 3, 2019 · 5:17 PM UTC

3
1
Replying to @davidbaron @annevk @MarcosC @plhw3org
Yes, we do.... It keeps getting deferred....
Replying to @davidbaron @annevk @MarcosC @plhw3org
@jgraham was meant to be working on that, IIRC; @AutomatedTester would know I presume? I know Chromium esp. had problems with perceptions of it being too spammy when they started doing so?
1
The trick is to figure out how to send the important notifications and not send the spammy notifications. I'm not saying its easy, but getting it right would be really useful!
2
1
more replies
Replying to @davidbaron @annevk @MarcosC @plhw3org
A coarse filtering of notifications per directory might be good enough. Ignore directories for specs we don't implement.