Skip to content

src: optimize NormalizeString to cut allocations and copies#63753

Open
whoekage wants to merge 1 commit into
nodejs:mainfrom
whoekage:perf/path-cc-normalizestring
Open

src: optimize NormalizeString to cut allocations and copies#63753
whoekage wants to merge 1 commit into
nodejs:mainfrom
whoekage:perf/path-cc-normalizestring

Conversation

@whoekage
Copy link
Copy Markdown

@whoekage whoekage commented Jun 5, 2026

What this changes

NormalizeString() in src/path.cc is the lexical path-normalization core
behind PathResolve(), NormalizeFileURLOrPath() and ToNamespacedPath().
It collapses //, removes ., resolves .., and strips trailing
separators, and it runs during module and package resolution, the compile
cache, --permission fs checks, and (on Windows) on every fs syscall via
ToNamespacedPath().

This rewrites the .. handling to a write-position rewind with a dotdot
floor index, the approach used by Go's path/filepath.Clean, and switches
segment building to in-place appends.

Before

For every .. segment the old code:

  • ran res = res.substr(0, idx), which allocated a new string and copied the
    entire surviving prefix (O(n²) copy volume on deep-backtrack paths);
  • ran two find_last_of(separator) scans, one to truncate and one to
    recompute lastSegmentLength.

For every normal segment it built two or three temporary std::strings
(std::string(separator) + std::string(path.substr(...))), and res was
never reserve()d.

After

  • A dotdot floor index marks where backtracking must stop. Resolving ..
    rewinds the write position to the previous separator and calls
    res.resize(), so there is no allocation, no prefix copy and no
    find_last_of.
  • Segments are written with push_back() and append(ptr, count), with no
    temporary strings.
  • res.reserve(path.size()) is called once (the output is always no longer
    than the input).
  • lastSegmentLength and the "does res already end with .." check are
    gone, subsumed by dotdot.

The change is 30 insertions and 33 deletions in src/path.cc.

Benchmarks

Measured with an A/B harness that compiles the old and new implementations
into a single binary (clang -O2, Apple M-series), corroborated by a cctest
timing run. Time per call:

input before after speedup
/usr/local/bin/node (typical) 71 ns 37 ns 1.9x
deep backtrack 222 ns 94 ns 2.4x
200 segments, no backtrack 3981 ns 1647 ns 2.4x
50x a/ + 50x ../ 2410 ns 522 ns 4.6x

Heap allocations per call, counted with an operator new override: the
50x a/ + 50x ../ case drops from 41 to 1, the 200-segment path from 6 to 1,
and typical or short paths stay at 0 (small-string optimization).

Correctness

The output is byte for byte identical to the previous implementation. I
verified this with a differential test of 1.2M random inputs (length 0 to 40
over {'/','\\','.','a','b','c'}, both allowAboveRoot values) against a
frozen copy of the old implementation, under both POSIX semantics (separator
/) and Windows semantics (IsPathSeparator matching / and \, with
separators / and \): 0 mismatches. The existing cctest PathResolve
cases pass, and new cases lock the touched behaviors (deep backtrack to root,
the .. floor with allowAboveRoot, and ., // and trailing-separator
collapse).

Scope

This is a hot helper, but on POSIX it is a small fraction of end-to-end JS
workloads. The main beneficiaries are path-resolution-heavy code and, on
Windows, fs operations, where ToNamespacedPath runs on every syscall. The
change is purely a constant-factor and allocation improvement with identical
observable behavior.

Prior art: Go path/filepath.Clean (lazybuf plus a dotdot index). A
related allocation optimization in this subsystem that was accepted is #50288.

Checklist
  • make -j4 test (UNIX) passes for the touched area (cctest PathTest*)
  • tests are included
  • documentation is changed or added
  • commit message follows commit guidelines

Rewrite the parent-dir ("..") handling in NormalizeString() to use a
write-position rewind with a "dotdot" floor index, the approach used by
Go's path/filepath.Clean, and build segments with in-place appends instead
of temporary-string concatenation.

Previously each ".." did `res = res.substr(0, idx)`, which allocated a new
string and copied the whole surviving prefix (O(n^2) copy volume on
deep-backtrack paths), plus two find_last_of() scans. Each normal segment
built two or three temporary std::strings, and res was never reserve()d.

Now ".." rewinds the write position to the previous separator and resize()s,
so there is no allocation, no prefix copy and no find_last_of call. Segments
are written with push_back() and append(). The output is byte for byte
identical to the previous implementation.
@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. labels Jun 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants