gh-142006: Fix HeaderWriteError in email.policy.default caused by extra newline #142008

pareshjoshij · 2025-11-27T10:37:39Z

Description

This PR fixes gh-142006 where email.policy.default would raise a HeaderWriteError when folding certain long address headers.

The Issue:
In _header_value_parser.py, the function _steal_trailing_WSP_if_exists moves trailing whitespace from the previous line to the current one. However, if the previous line contained only that whitespace, it was left as an empty string ''. When joined, this resulted in a double newline (\n\n), which is illegal in headers and triggered the error.

The Fix:
Modified _steal_trailing_WSP_if_exists to remove the line entirely (lines.pop()) if it becomes empty after stealing the whitespace.

Verification:
Added a regression test test_fold_address_list_with_stolen_whitespace to Lib/test/test_email/test_policy.py which reproduces the reported crash and verifies the fix.

Linked Issue

Fixes gh-142006

bitdancer

Thanks for finding and fixing this bug.

Lib/email/_header_value_parser.py

Lib/test/test_email/test_policy.py

Misc/NEWS.d/next/Library/2025-11-27-10-49-13.gh-issue-142006.nzJDG5.rst

bedevere-app · 2025-11-27T18:12:50Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

pareshjoshij · 2025-11-28T05:11:24Z

Thanks @bitdancer! I have made the requested changes; please review again.

Updated the comment style in _header_value_parser.py.
Moved the test to test__header_value_parser.py inside TestFolding, refactored to use max_line_length, and switched to the file's native self._test style.
Updated the NEWS entry text as suggested.

Please review again.

pareshjoshij · 2025-11-28T05:23:00Z

I have made the requested changes; please review again.

I have made the requested changes; please review again.

bedevere-app · 2025-11-28T05:23:05Z

Thanks for making the requested changes!

@bitdancer: please review the changes made to this pull request.

bitdancer

Test needs a tweak.

Lib/test/test_email/test__header_value_parser.py

bitdancer · 2025-12-05T20:38:31Z

Lib/test/test_email/test__header_value_parser.py

+        # The "long" part (20 chars) forces the wrap because 20 > 10.
+        text = "a, " + ("b" * 20)
+        expected = "a,\n " + ("b" * 20) + "\n"
+


I'd drop the blank lines. They don't add anything and most other tests don't have internal blank lines.

Misc/NEWS.d/next/Library/2025-11-27-10-49-13.gh-issue-142006.nzJDG5.rst

bedevere-app · 2025-12-05T20:44:46Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

pareshjoshij · 2025-12-06T02:26:24Z

I have made the requested changes; please review again.

bedevere-app · 2025-12-06T02:26:29Z

Thanks for making the requested changes!

@bitdancer: please review the changes made to this pull request.

bitdancer

LGTM

miss-islington-app · 2025-12-06T20:59:38Z

Thanks @pareshjoshij for the PR, and @bitdancer for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

…by extra newline (pythonGH-142008) RDM: This fixes a subtle folding error that showed up when a token exactly filled a line and was followed by whitespace and a token with no folding whitespace that was longer than a line. In this particular circumstance the whitespace after the first token got pushed on to the next line, and then stolen to go in front of the next unfoldable token...leaving a completely empty line in the line buffer. That line got turned in to a newline, which is RFC illegal, and the newish security check caught it. The fix is to just delete that empty line from the buffer. (cherry picked from commit 07eff89) Co-authored-by: Paresh Joshi <[email protected]> Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>

bedevere-app · 2025-12-06T20:59:49Z

GH-142361 is a backport of this pull request to the 3.14 branch.

bedevere-app · 2025-12-06T20:59:53Z

GH-142362 is a backport of this pull request to the 3.13 branch.

… by extra newline (GH-142008) (#142361) gh-142006: Fix HeaderWriteError in email.policy.default caused by extra newline (GH-142008) RDM: This fixes a subtle folding error that showed up when a token exactly filled a line and was followed by whitespace and a token with no folding whitespace that was longer than a line. In this particular circumstance the whitespace after the first token got pushed on to the next line, and then stolen to go in front of the next unfoldable token...leaving a completely empty line in the line buffer. That line got turned in to a newline, which is RFC illegal, and the newish security check caught it. The fix is to just delete that empty line from the buffer. (cherry picked from commit 07eff89) Co-authored-by: Paresh Joshi <[email protected]> Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>

… by extra newline (GH-142008) (#142362) gh-142006: Fix HeaderWriteError in email.policy.default caused by extra newline (GH-142008) RDM: This fixes a subtle folding error that showed up when a token exactly filled a line and was followed by whitespace and a token with no folding whitespace that was longer than a line. In this particular circumstance the whitespace after the first token got pushed on to the next line, and then stolen to go in front of the next unfoldable token...leaving a completely empty line in the line buffer. That line got turned in to a newline, which is RFC illegal, and the newish security check caught it. The fix is to just delete that empty line from the buffer. (cherry picked from commit 07eff89) Co-authored-by: Paresh Joshi <[email protected]> Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>

pareshjoshij requested review from a team and gaogaotiantian as code owners November 27, 2025 10:37

bedevere-app bot added the awaiting review label Nov 27, 2025

bedevere-app bot mentioned this pull request Nov 27, 2025

email.policy.default adds an extra newline char to email address headers and raises HeaderWriteError on stringify #142006

Closed

Fix pythongh-142006: email.policy.default adds extra newline to headers

328bbac

pareshjoshij force-pushed the fix-gh-142006 branch from 42d48ed to 328bbac Compare November 27, 2025 10:44

📜🤖 Added by blurb_it.

45d7830

bitdancer requested changes Nov 27, 2025

View reviewed changes

Lib/email/_header_value_parser.py Outdated Show resolved Hide resolved

Lib/test/test_email/test_policy.py Outdated Show resolved Hide resolved

Misc/NEWS.d/next/Library/2025-11-27-10-49-13.gh-issue-142006.nzJDG5.rst Outdated Show resolved Hide resolved

bedevere-app bot added awaiting changes and removed awaiting review labels Nov 27, 2025

pareshjoshij added 2 commits November 28, 2025 10:28

Address review: fix comment style and move test

8447ed5

Merge news file and update text

7d63071

pareshjoshij requested a review from bitdancer November 28, 2025 05:09

bedevere-app bot added awaiting change review and removed awaiting changes labels Nov 28, 2025

bitdancer requested changes Dec 5, 2025

View reviewed changes

bedevere-app bot added awaiting changes and removed awaiting change review labels Dec 5, 2025

Fix test logic and comments per review

55e00ae

bedevere-app bot added awaiting change review and removed awaiting changes labels Dec 6, 2025

bedevere-app bot requested a review from bitdancer December 6, 2025 02:26

bitdancer approved these changes Dec 6, 2025

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting change review labels Dec 6, 2025

bitdancer added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels Dec 6, 2025

bitdancer merged commit 07eff89 into python:main Dec 6, 2025
55 of 58 checks passed

bedevere-app bot removed the awaiting merge label Dec 6, 2025

bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label Dec 6, 2025

bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Dec 6, 2025

pareshjoshij deleted the fix-gh-142006 branch December 7, 2025 00:01

Uh oh!

gh-142006: Fix HeaderWriteError in email.policy.default caused by extra newline #142008

gh-142006: Fix HeaderWriteError in email.policy.default caused by extra newline #142008

Conversation

pareshjoshij commented Nov 27, 2025

Description

Linked Issue

Uh oh!

bitdancer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bedevere-app bot commented Nov 27, 2025

Uh oh!

pareshjoshij commented Nov 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pareshjoshij commented Nov 28, 2025

Uh oh!

bedevere-app bot commented Nov 28, 2025

Uh oh!

bitdancer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bitdancer Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bedevere-app bot commented Dec 5, 2025

Uh oh!

pareshjoshij commented Dec 6, 2025

Uh oh!

bedevere-app bot commented Dec 6, 2025

Uh oh!

bitdancer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-app bot commented Dec 6, 2025

Uh oh!

bedevere-app bot commented Dec 6, 2025

Uh oh!

bedevere-app bot commented Dec 6, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pareshjoshij commented Nov 28, 2025 •

edited

Loading