Skip to content

readline: add unicode line separator option#63789

Open
scarab-systems wants to merge 1 commit into
nodejs:mainfrom
scarab-systems:fix/readline-line-separator-option
Open

readline: add unicode line separator option#63789
scarab-systems wants to merge 1 commit into
nodejs:mainfrom
scarab-systems:fix/readline-line-separator-option

Conversation

@scarab-systems
Copy link
Copy Markdown

Fixes: #22448

This adds a unicodeLineSeparators option to readline.createInterface() and readlinePromises.createInterface().

By default, behavior is unchanged: \u2028 and \u2029 continue to be treated as line endings. When unicodeLineSeparators: false is passed, readline only splits on CR, LF, and CRLF, allowing formats such as JSONL to preserve Unicode line and paragraph separators inside record contents.

Tests added/updated:

  • ./node test/parallel/test-readline-line-separators.js
  • python3 tools/test.py --shell ./node test/parallel/test-readline-line-separators.js
  • make lint-js LINT_JS_TARGETS="lib/internal/readline/interface.js lib/readline.js test/parallel/test-readline-line-separators.js"
  • make lint-md LINT_MD_FILES="doc/api/readline.md"

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. readline Issues and PRs related to the built-in readline module. labels Jun 8, 2026
Signed-off-by: Scarab Systems <scarab.systems@yahoo.com>
@scarab-systems scarab-systems force-pushed the fix/readline-line-separator-option branch from 90cb9fa to d936651 Compare June 8, 2026 01:12
Comment thread doc/api/readline.md
Comment on lines +725 to +726
* `unicodeLineSeparators` {boolean} If `true`, `\u2028` and `\u2029` will be
treated as end-of-line input. **Default:** `true`.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if a boolean is the correct way, or if we should instead let the user define their own list of separator as suggested in #60606 (comment)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense. I agree a boolean may be too narrow if we are adding a new API surface.

Would an additionalLineSeparators option be a better shape here? My thinking is that \n, \r, and \r\n should remain the fixed readline line endings, while this option controls only the extra separators. The default could preserve current behavior with ['\u2028', '\u2029'], and JSONL users could pass [] to preserve U+2028/U+2029 inside records.

I can rework the PR in that direction if that seems aligned with what you had in mind.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that reverting back to only \n/\r is likely a goal, though that would be a semver change, so we'd better land that separately. We could add a option lineSeparators that would take either a string, a regex, and maybe some magic values (e.g. Symbol.for('nodejs.readline.classic'), Symbol.for('nodejs.readline.unicode'))

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense. I can rework this PR away from unicodeLineSeparators and toward a lineSeparators option, while preserving the current default behavior here and leaving any default revert to a separate semver-major change.

For this PR, I’ll aim to keep the immediate problem covered by two built-in modes: current Unicode behavior by default, and a classic CR/LF/CRLF-only mode for JSONL-style inputs. If you think custom string/RegExp separators should land in the same PR rather than as a follow-up, I can shape it that way too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-ci PRs that need a full CI run. readline Issues and PRs related to the built-in readline module.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

readline: processing \u2028 and \u2029

3 participants