Repository navigation
fix: report the right line and column for CRLF and mixed line endings - #642
Conversation
|
The one failing check here is not from this change. The same job fails identically on other open PRs that do not touch parsing — run 38073613876 on Every other check passes: the 3.9-3.13 test matrix on Ubuntu, macOS and Windows, and the five other integration jobs. Failure log: https://github.com/python-poetry/tomlkit/actions/runs/38060239938/job/114236854801 |
|
The This branch's own |
|
This red is not from this change. The failing test is Poetry's own It looks like the leg broke for everyone around 10-08: master's run of the same workflow was still green on 10-07 (https://github.com/python-poetry/tomlkit/actions/runs/37619670798), and in the workflow list every pull-request Integration run since 10-09 is red. I read one of them on an unrelated branch, Flagging it rather than rebasing over it — the Poetry-side test looks like it needs a pin or a Python-patch-dependent skip, which I would not want to decide inside this PR. |
Summary
The line and column reported in a parse error depend on the document's line endings.
Source._to_linecolsplits the source withsplitlines(), which drops the terminators, and then assumes every break costs exactly one character:A
\r\nbreak costs two, so the running offset falls one character behind for every line before the error, and the error lands on the wrong line and the wrong column.exceptions.pydocuments that aParseError"references the line and location within the line where the error was encountered", andtoml_file.py:36-44only normalises\r\nwhen the whole document uses it consistently — so a file with mixed endings read from disk reaches this path today.The fix lets each split line carry its own terminator (
splitlines(keepends=True)) and uses that length, so the offset matches the characters actually consumed. Documents that use only\ncome out identical; CRLF and mixed documents now report the position the LF form already did.Test:
pytest tests/test_parser.py -k positionfails on main (858186a) — the CRLF case reportsUnexpectedCharError("Unexpected character: '@' at line 4 col 0")where the character is on line 3, column 4 — and passes on this branch. The full suite (pytest --ignore=tests/test_toml_tests.py -q) is 420 passed, andruff check/ruff format --checkat the version pinned in.pre-commit-config.yaml(0.16.10) are clean. I also walked every character index of every file undertests/examplesand compared the reported position before and after: 6913 positions disagreed for CRLF copies of those documents and 0 disagree after the change, while LF-only documents show 0 differences either way.Not run:
tests/test_toml_tests.py, because thetests/toml-testsubmodule is empty in this checkout. I read it — it assertsTOMLKitErroris raised for invalid documents and compares parsed JSON for valid ones, and does not look at line/column — but that is a reading, not a run.Agent Drafting Metadata