aboutsummaryrefslogtreecommitdiffstatshomepage
path: root/Lib/email/_header_value_parser.py
Commit message (Collapse)AuthorAge
* gh-121284: Fix email address header folding with parsed encoded-word (GH-122754)Mike Edmunds2025-03-18
| | | | | | | | | | | | | | | Email generators using email.policy.default may convert an RFC 2047 encoded-word to unencoded form during header refolding. In a structured header, this could allow 'specials' chars outside a quoted-string, leading to invalid address headers and enabling spoofing. This change ensures a parsed encoded-word that contains specials is kept as an encoded-word while the header is refolded. [Better fix from @bitdancer.] --------- Co-authored-by: R David Murray <rdmurray@bitdance.com> Co-authored-by: Petr Viktorin <encukou@gmail.com>
* gh-80222: Fix email address header folding with long quoted-string (#122753)Mike Edmunds2025-01-18
| | | | | | | Email generators using email.policy.default could incorrectly omit the quote ('"') characters from a quoted-string during header refolding, leading to invalid address headers and enabling header spoofing. This change restores the quote characters on a bare-quoted-string as the header is refolded, and escapes backslash and quote chars in the string.
* gh-121650: Encode newlines in headers, and verify headers are sound (GH-122233)Petr Viktorin2024-07-31
| | | | | | | | | | | | | | | | | | | | | | | | | ## Encode header parts that contain newlines Per RFC 2047: > [...] these encoding schemes allow the > encoding of arbitrary octet values, mail readers that implement this > decoding should also ensure that display of the decoded data on the > recipient's terminal will not cause unwanted side-effects It seems that the "quoted-word" scheme is a valid way to include a newline character in a header value, just like we already allow undecodable bytes or control characters. They do need to be properly quoted when serialized to text, though. ## Verify that email headers are well-formed This should fail for custom fold() implementations that aren't careful about newlines. Co-authored-by: Bas Bloemsaat <bas@bloemsaat.org> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* gh-120930: Remove extra blank occuring in wrapped encoded words in email ↵Matthieu Caneill2024-07-18
| | | | headers (GH-121747)
* gh-118643: Fix AttributeError in the email module (GH-119099)Serhiy Storchaka2024-05-22
| | | | | | | | Fix regression introduced in gh-100884: AttributeError when re-fold a long address list. Also fix more cases of incorrect encoding of the address separator in the address list missed in gh-100884.
* gh-92081: Fix for email.generator.Generator with whitespace between encoded ↵Toshio Kuratomi2024-05-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | words. (#92281) * Fix for email.generator.Generator with whitespace between encoded words. email.generator.Generator currently does not handle whitespace between encoded words correctly when the encoded words span multiple lines. The current generator will create an encoded word for each line. If the end of the line happens to correspond with the end real word in the plaintext, the generator will place an unencoded space at the start of the subsequent lines to represent the whitespace between the plaintext words. A compliant decoder will strip all the whitespace from between two encoded words which leads to missing spaces in the round-tripped output. The fix for this is to make sure that whitespace between two encoded words ends up inside of one or the other of the encoded words. This fix places the space inside of the second encoded word. A second problem happens with continuation lines. A continuation line that starts with whitespace and is followed by a non-encoded word is fine because the newline between such continuation lines is defined as condensing to a single space character. When the continuation line starts with whitespace followed by an encoded word, however, the RFCs specify that the word is run together with the encoded word on the previous line. This is because normal words are filded on syntactic breaks by encoded words are not. The solution to this is to add the whitespace to the start of the encoded word on the continuation line. Test cases are from #92081 * Rename a variable so it's not confused with the final variable.
* bpo-40944: Fix IndexError when parse emails with truncated Message-ID, ↵Ivan Savin2024-04-17
| | | | | address, routes, etc (GH-20790) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* gh-86650: Fix IndexError when parse emails with invalid Message-ID (GH-117934)Serhiy Storchaka2024-04-17
| | | | | | | In particularly, one-off addresses generated by Microsoft Outlook: https://learn.microsoft.com/en-us/office/client-developer/outlook/mapi/one-off-addresses Co-authored-by: fsc-eriker <72394365+fsc-eriker@users.noreply.github.com>
* gh-75171: Fix parsing invalid email address headers starting or ending with ↵tsufeki2024-04-17
| | | | | | a dot (GH-15600) Co-authored-by: Tim Bell <timothybell@gmail.com> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* gh-100884: email/_header_value_parser: don't encode list separators (GH-100885)Thomas Weißschuh2024-02-17
| | | | | ListSeparator should not be encoded. This could happen when a long line pushes its separator to the next line, which would have been encoded.
* gh-113594: Fix UnicodeEncodeError in TokenList.fold() (GH-113730)Serhiy Storchaka2024-01-10
| | | It occurred when try to re-encode an unknown-8bit part combined with non-unknown-8bit part.
* gh-102498 Clean up unused variables and imports in the email module (#102482)JosephSBoyle2023-04-24
| | | | | | | | | | | | | | | | | | | | | * Clean up unused variables and imports in the email module * Remove extra newline char * Remove superflous dict+unpacking syntax * Remove unused 'msg' var * Clean up unused variables and imports in the email module * Remove extra newline char * Remove superflous dict+unpacking syntax * Remove unused 'msg' var --------- Co-authored-by: Barry Warsaw <barry@python.org>
* gh-93010: InvalidHeaderError used but nonexistent (#93015)oda-gitso2022-05-23
| | | | | * fix issue 93010 Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
* bpo-43323: Fix UnicodeEncodeError in the email module (GH-32137)Serhiy Storchaka2022-04-30
| | | | | It was raised if the charset itself contains characters not encodable in UTF-8 (in particular \udcxx characters representing non-decodable bytes in the source).
* bpo-39040: Fix parsing of email mime headers with whitespace between ↵Abhilash Raj2020-05-28
| | | | | | | | | | | | | | | encoded-words. (gh-17620) * bpo-39040: Fix parsing of email headers with encoded-words inside a quoted string. It is fairly common to find malformed mime headers (especially content-disposition headers) where the parameter values, instead of being encoded to RFC standards, are "encoded" by doing RFC 2047 "encoded word" encoding, and then enclosing the whole thing in quotes. The processing of these malformed headers was incorrectly leaving the spaces between encoded words in the decoded text (whitespace between adjacent encoded words is supposed to be stripped on decoding). This changeset fixes the encoded word processing inside quoted strings (bare-quoted-string) to do correct RFC 2047 decoding by stripping that whitespace.
* bpo-38708: email: Fix a potential IndexError when parsing Message-ID (GH-17504)Abhilash Raj2019-12-08
| | | | Fix a potential IndexError when passing an empty value to the message-id parser. Instead, HeaderParseError should be raised.
* bpo-38698: Add a new InvalidMessageID token to email header parser. (GH-17503)Abhilash Raj2019-12-08
| | | | This adds a new InvalidMessageID token to the email header parser which can be used to represent invalid message-id headers in the parse tree.
* bpo-38698: Prevent UnboundLocalError to pop up in parse_message_id (GH-17277)Claudiu Popa2019-12-04
| | | | | | | | | parse_message_id() was improperly using a token defined inside an exception handler, which was raising `UnboundLocalError` on parsing an invalid value. https://bugs.python.org/issue38698
* bpo-38332: Catch KeyError from unknown cte in encoded-word. (GH-16503)Andrei Troie2019-10-05
| | | KeyError should cause a failure in parsing the encoded word and should be caught and raised as a _InvalidEWError instead.
* bpo-37764: Fix infinite loop when parsing unstructured email headers. (GH-15239)Ashwin Ramaswami2019-08-31
| | | | | | | | | | | | Fixes a case in which email._header_value_parser.get_unstructured hangs the system for some invalid headers. This covers the cases in which the header contains either: - a case without trailing whitespace - an invalid encoded word https://bugs.python.org/issue37764 This fix should also be backported to 3.7 and 3.8 https://bugs.python.org/issue37764
* bpo-37482: Fix email address name with encoded words and special chars ↵bsiem2019-08-21
| | | | | | | | | | | | | | | | | | | | | | | (GH-14561) Special characters in email address header display names are normally put within double quotes. However, encoded words (=?charset?x?...?=) are not allowed withing double quotes. When the header contains a word with special characters and another word that must be encoded, the first one must also be encoded. In the next example, the display name in the From header is quoted and therefore the comma is allowed; in the To header, the comma is not within quotes and not encoded, which is not allowed and therefore rejected by some mail servers. From: "Foo Bar, France" <foo@example.com> To: Foo Bar, =?utf-8?q?Espa=C3=B1a?= <foo@example.com> https://bugs.python.org/issue37482
* bpo-32178: Fix IndexError trying to parse 'To' header starting with ':'. ↵Abhilash Raj2019-08-11
| | | | | | | | | | (GH-15044) This should fix the IndexError trying to retrieve `DisplayName.display_name` and `DisplayName.value` when the `value` is basically an empty string. https://bugs.python.org/issue32178
* bpo-34155: Dont parse domains containing @ (GH-13079)jpic2019-07-17
| | | | | | | | | | | | | | | | | | | | | | Before: >>> email.message_from_string('From: a@malicious.org@important.com', policy=email.policy.default)['from'].addresses (Address(display_name='', username='a', domain='malicious.org'),) >>> parseaddr('a@malicious.org@important.com') ('', 'a@malicious.org') After: >>> email.message_from_string('From: a@malicious.org@important.com', policy=email.policy.default)['from'].addresses (Address(display_name='', username='', domain=''),) >>> parseaddr('a@malicious.org@important.com') ('', 'a@') https://bugs.python.org/issue34155
* Fix IndexError when parsing unexpectedly ending quoted-string. (GH-14813)Abhilash Raj2019-07-17
| | | | This exception was caused because the input ended unexpectedly with only one single quote instead of a pair with some value inside it.
* bpo-37461: Fix infinite loop in parsing of specially crafted email headers ↵Abhilash Raj2019-07-17
| | | | | | | | | | (GH-14794) * bpo-37461: Fix infinite loop in parsing of specially crafted email headers. Some crafted email header would cause the get_parameter method to run in an infinite loop causing a DoS attack surface when parsing those headers. This patch fixes that by making sure the DQUOTE character is handled to prevent going into an infinite loop.
* Fix infinite loop in email folding logic (GH-12732)Paul Ganssle2019-07-16
| | | | | | | | | | As far as I can tell, this infinite loop would be triggered if: 1. The value being folded contains a single word (no spaces) longer than max_line_length 2. The max_line_length is shorter than the encoding's name + 9 characters. bpo-36564: https://bugs.python.org/issue36564
* bpo-29412: Fix indexError when parsing a header value ending unexpectedly ↵Abhilash Raj2019-06-26
| | | | | | | (GH-14387) * patched string index out of range error in get_word function of _header_value_parser.py and created tests in test__header_value_parser.py for CFWS. * Raise HeaderParseError instead of continuing when parsing a word.
* bpo-36520: Email header folded incorrectly (#13608)websurfer52019-06-06
| | | | | | | | | | | | | | | | | * bpo-36520: reset the encoded word offset when starting a new line during an email header folding operation * 📜🤖 Added by blurb_it. * bpo-36520: add an additional test case, and provide descriptive comments for the test_folding_with_utf8_encoding_* tests * bpo-36520: fix whitespace issue * bpo-36520: changes per reviewer request -- remove extraneous backslashes; add whitespace between terminating quotes and line-continuation backslashes; use "bpo-" instead of "issue #" in comments
* bpo-21315: Fix parsing of encoded words with missing leading ws. (#13425)Abhilash Raj2019-06-05
| | | | | | | | | | | | * bpo-21315: Fix parsing of encoded words with missing leading ws. Because of missing leading whitespace, encoded word would get parsed as unstructured token. This patch fixes that by looking for encoded words when splitting tokens with whitespace. Missing trailing whitespace around encoded word now register a defect instead. Original patch suggestion by David R. Murray on bpo-21315.
* bpo-35805: Add parser for Message-ID email header. (GH-13397)Abhilash Raj2019-06-04
| | | | | | | | | | | | | * bpo-35805: Add parser for Message-ID header. This parser is based on the definition of Identification Fields from RFC 5322 Sec 3.6.4. This should also prevent folding of Message-ID header using RFC 2047 encoded words and hence fix bpo-35805. * Prevent folding of non-ascii message-id headers. * Add fold method to MsgID token to prevent folding.
* Fix typos in docs and docstrings (GH-13745)Xtreak2019-06-03
|
* bpo-33524: Fix the folding of email header when max_line_length is 0 or None ↵Abhilash Raj2019-05-17
| | | | | (#13391) and there are non-ascii characters in the header.
* bpo-33529, email: Fix infinite loop in email header encoding (GH-12020)Krzysztof Wojcik2019-05-14
|
* bpo-34424: Handle different policy.linesep lengths correctly. (#8803)Jens Troeger2019-05-13
|
* email: use dict instead of OrderedDict (GH-11709)Inada Naoki2019-02-05
|
* bpo-35133: Fix mistakes when concatenate string literals on different lines. ↵Serhiy Storchaka2018-11-05
| | | | | | | | | | (GH-10284) Two kind of mistakes: 1. Missed space. After concatenating there is no space between words. 2. Missed comma. Causes unintentional concatenating in a list of strings.
* bpo-33476: Fix _header_value_parser when address group is missing final ';' ↵Dong-hee Na2018-07-28
| | | | (GH-7484)
* bpo-32746: Fix multiple typos (GH-5144)Leo Arias2018-02-03
| | | Fix typos found by codespell in docs, docstrings, and comments.
* bpo-27931: Fix email address header parsing error (#5329)jayyyin2018-01-29
| | | | Correctly handle addresses whose username is an empty quoted string.
* bpo-27240 Rewrite the email header folding algorithm. (#3488)R. David Murray2017-12-03
| | | | | | | | | | | | | | | | | | | | | The original algorithm tried to delegate the folding to the tokens so that those tokens whose folding rules differed could specify the differences. However, this resulted in a lot of duplicated code because most of the rules were the same. The new algorithm moves all folding logic into a set of functions external to the token classes, but puts the information about which tokens can be folded in which ways on the tokens...with the exception of mime-parameters, which are a special case (which was not even implemented in the old folder). This algorithm can still probably be improved and hopefully simplified somewhat. Note that some of the test expectations are changed. I believe the changes are toward more desirable and consistent behavior: in general when (re) folding a line the canonical version of the tokens is generated, rather than preserving errors or extra whitespace.
* bpo-30349: Raise FutureWarning for nested sets and set operations (#1553)Serhiy Storchaka2017-11-16
| | | | in regular expressions.
* bpo-30532: Fix whitespace folding in certain casesJoel Hillacre2017-06-26
| | | Leading whitespace was incorrectly dropped during folding of certain lines in the _header_value_parser's folding algorithm. This makes the whitespace handling code consistent.
* #27364: fix "incorrect" uses of escape character in the stdlib.R David Murray2016-09-08
| | | | | | | And most of the tools. Patch by Emanual Barry, reviewed by me, Serhiy Storchaka, and Martin Panter.
* Issues #23147, #23148: Presumably fixed bugs in folding UnstructuredTokenList.Serhiy Storchaka2016-07-17
|
* Issue #27076: Doc, comment and tests spelling fixesMartin Panter2016-05-26
| | | | Most fixes to Doc/ and Lib/ directories by Ville Skyttä.
* Issue #26778: Fixed "a/an/and" typos in code comment and documentation.Serhiy Storchaka2016-04-17
|
* #24211: Add RFC6532 support to the email library.R David Murray2015-05-17
| | | | | | | | | This could use more edge case tests, but the basic functionality is tested. (Note that this changeset does not add tailored support for the RFC 6532 message/global MIME type, but the email package generic facilities will handle it.) Reviewed by Maciej Szulik.
* #23745: handle duplicate MIME parameter names in new parser.R David Murray2015-03-29
| | | | | | | | | | This mimics get_param's error handling for the most part. It is slightly better in some regards as get_param can produce some really weird results for duplicate *0* parts. It departs from get_param slightly in that if we have a mix of non-extended and extended pieces for the same parameter name, the new parser assumes they were all supposed to be extended and concatenates all the values, whereas get_param always picks the non-extended parameter value. All of this error recovery is pretty much arbitrary decisions...
* #20977: fix undefined name in the email module. Patch by Rose Ames.Ezio Melotti2014-08-04
|
* Issue #20976: pyflakes: Remove unused importsVictor Stinner2014-03-20
|
generated by cgit v1.2.3 (git 2.39.1) at 2025-05-04 17:54:58 +0000