synchronous/asynchronous error events and state changes on HTMLMediaElement

[karlt]

What is the issue with the HTML Standard?

"error" events, in both the dedicated media source failure steps and the media data processing steps list are described as being fired synchronously, but implementations queue a task to fire the events.

Implementations implement the error events in the media data processing steps as specified prior to
22d0fa1#diff-42692259bbea922a55e8e0f8a8b7860df9b59e29b3f955bcfd4d0a3ca852cc01R27539, which didn't reflect the behavior of implementations.

Implementations have been unkeen to fire events synchronously because doing so is harder to get right. The event usually must be the last action in the task. For example, resetting the delaying-the-load-event flag after synchronously firing the event, as specified, would not have the desired effect when an error event handler triggers the load algorithm again, setting the flag. Implementations have additional object lifetime concerns that are not obvious from the spec prose. See for example WebKit/WebKit@3582cc1 and https://bugzilla.mozilla.org/show_bug.cgi?id=476771.

Implementations implement the dedicated media source failure steps similarly to what was specified prior to b64ac15#diff-42692259bbea922a55e8e0f8a8b7860df9b59e29b3f955bcfd4d0a3ca852cc01L27234
That spec change changed to the description of a number of state changes to asynchronous, which also didn't reflect behavior of implementations. That meant that state changes in response to MediaSource#endOfStream(), such as setting the error attribute, would be asynchronous when the readyState attribute equals HAVE_NOTHING, which would be inconsistent with similar state changes when the readyState attribute is greater than HAVE_NOTHING.

[karlt]

Delays in state changes are also observable when the state changes are triggered on Failure during the resource selection algorithm synchronous steps. When "Failed with attribute" is reached, either because "src attribute's value is the empty string" or because urlRecord is failure, the state changes need not be asynchronous. They are synchronous in Blink and Webkit. They are only asynchronous in Gecko due to a web platform test for the async state changes described above that were not implemented in other engines.

When "src attribute's value is the empty string", the synchronous section ends, but there is no instruction to run the remaining steps in parallel. When urlRecord is failure, the state changes are currently specified as "steps in parallel", which is contrary to the synchronous implementations.

[karlt]

Proposal:

1. Revert the effects of b64ac15#diff-42692259bbea922a55e8e0f8a8b7860df9b59e29b3f955bcfd4d0a3ca852cc01L27234 on the dedicated media source failure steps and their invocation.
2. Revert the effects of 22d0fa1#diff-42692259bbea922a55e8e0f8a8b7860df9b59e29b3f955bcfd4d0a3ca852cc01R27539 on the events named error, abort, and emptied, and on the associated state changes, in the media data processing steps list.
3. When Failure occurs during the resource selection algorithm synchronous steps, invoke the dedicated media source failure steps directly, without any parallel steps.

The third step is consistent with Blink and Webkit, but not current Gecko versions. Steps 1 and 2 are consistent with all three engines.

[annevk]

Do we have test coverage for this? I ask because I tried to fix a related case in #7855, but I got stuck on writing a good test (perhaps all it takes is making WebDriver click activate the document?).

cc @whatwg/media

[karlt]

The following tests are expecting a synchronous error event when the delaying-the-load-event flag is reset. They do not pass in Blink. They pass in Webkit, not because the error event is synchronous, but apparently other factors are delaying the load event until after the error event. They will (usually) fail in Gecko after proposal item 3 is restored there.
resource-selection-invoke-audio-constructor.html
resource-selection-invoke-set-src-not-in-document.html
resource-selection-invoke-set-src.html

https://phabricator.services.mozilla.com/D242880 adds some more tests, which are currently a mix of:

1. Testing for what all browsers do.
2. Testing what is specified and unaffected by changes here.
3. Testing for what is currently specified but not what browsers do, but some browsers happen to pass, not because they implement as specified, but because of other differences such as Blink not implementing the end of stream algorithm synchronously as specified.

The comments in the file aim to clarify.

I'm not clear what issue you were hitting with a test for #7855
Autoplay permission typically needs to be granted to run media tests live.
I guess wpt is somehow dealing with this through WebDriver because I haven't observed similar issues that way.