Newline parsing in H1

[annevk]

Popular clients handle LF, CR, and CRLF as newlines. HTTP only defines CRLF.

Previously: https://lists.w3.org/Archives/Public/ietf-http-wg/2014JulSep/0123.html.

[annevk]

To clarify, this is about the CRLF in the HTTP-message production.

[mnot]

Covered here:
http://httpwg.org/specs/rfc7230.html#message.robustness

... but I agree it could be more visible (e.g., in 3).

[annevk]

And clients also accept lone CR. It seems worth testing either that they reject (network error) or adding that.

[mnot]

@annevk see #68 - in situ, see https://httpwg.org/http-core/draft-ietf-httpbis-messaging-latest.html#message.parsing

I'm still not sure it's obvious enough to someone reading in isolation; thoughts?

[annevk]

@mnot that text does not treat CR without LF as a newline, unless I'm missing something. That text even seems to suggest that CRCRLF would be a single newline, whereas I'm pretty sure it ends up being treated as two newlines.

[wtarreau]

@annevk I'm pretty sure CR alone is not valid, because I remember such a discussion a few years ago by the time haproxy used to accept this, after which I removed support for this. CRCRLF could at best be considered as a CR at the end of a line, though normally it's not a valid character there. It may possibly be replaced by LWS though.

[wtarreau]

Just found it, its in RFC7230#3.5 : https://trac.tools.ietf.org/html/rfc7230#section-3.5

Although the line terminator for the start-line and header fields is
the sequence CRLF, a recipient MAY recognize a single LF as a line
terminator and ignore any preceding CR.

[annevk]

Right, I realize it's not valid, but I also know that clients nevertheless accept it, which is why I raised this.

[reschke]

All clients?

[mnot]

If they don't, they're not interoperable on the Web as we know it.

[royfielding]

The Web as we know it does not contain any CR-only line-delimited protocol services other than those deliberately attempting response smuggling. The protocol specifically forbids bare CR as a line delimiter, since that is known to not be interoperable with the vast majority of protocol parsers that exclusively check for LF and ignore CR. This was a big deal in 1994, but even then it was obvious that CR-delimited messages wouldn't make it through proxies. Nobody has objected to it since OS X was released and the last of the native Mac servers disappeared.

If some browsers interpret CR as a line delimiter in HTTP protocol fields (other than payload content), they have security holes that should be fixed. No change to the RFC is possible here; we can't make changes that introduce new security holes to previously compliant parsers.