upvote
The Windows filesystem isn't slow per se, it's a slowness caused by "a thousand cuts" type of problem.

https://github.com/Microsoft/WSL/issues/873#issuecomment-425...

reply
This is not due to slowness of the file system. Native ntfs tools are much faster than Unix ones in some situations. The issue is that running Unix software on windows will naturally have a performance impact. You see the same thing in reverse using Wine on Linux. Windows uses a different design for IO so requires software to be written with that design in mind.
reply
> Native ntfs tools are much faster than Unix ones in some situations. The issue is that running Unix software on windows will naturally have a performance impact. You see the same thing in reverse using Wine on Linux.

Not true. There are increasingly more cases where Windows software, written with Windows in mind and only tested on Windows, performs better atop Wine.

Sure, there are interface incompatibilities that naturally create performance penalties, but a lot of stuff maps 1:1, and Windows was historically designed to support multiple user-space ABIs; Win32 calls are broken down into native kernel calls by kernel32, advapi32, etc., for example, similar to how libc works on Unix-like operating systems.

reply
It's pretty typical these days for software, particularly games of the DX9-11 eras to perform better on Wine/Proton then they do under native Windows on the same hardware.
reply
[flagged]
reply
The file system isn't slow. The slowness will be present in any file system due to the file system filters that all file system calls pass though.
reply
Right, by “file system” here I mean all of the layers between the application talking in terms of named files and whatever first starts talking in terms of block addresses.

Also, as far as my (very limited) understanding goes, there are more architectural performance problems than just filters (and, to me, filters don’t necessarily sound like performance bankruptcy, provided the filter in question isn’t mandatory, un-removable Microsoft Defender). I seem to remember that path parsing is accomplished in NT by each handler chopping off the initial portion that it understands and passing the remaining suffix to the next one as an uninterpreted string (cf. COM monikers), unlike Unix where the slash-separated list is baked into the architecture, and the former design makes it much harder to have (what Unix calls) a “dentry cache” that would allow the kernel to look up meanings of popular names without going through the filesystem(s).

reply
NTFS will perform directory B+-tree lookups (this is where it walks the path) until it finds the requested file. The Cache Manager caches these B+-trees.

From there, it hits the MFT, finds the specific record for the file, loads the MFT record, and ultimately returns the FILE_OBJECT to the I/O Manager and it bubbles up the chain back to (presumably) Win32. The MFT is just a linear array of records, which include file and directories (directory records are just a record with directory = true, essentially).

Obviously simplified. Windows Internals will be your friend, if you want to know more.

reply
Thanks for the explanation! Linux, meanwhile, will[1] in the normal case walk a sequence[2] of hash tables (representing incomplete but up-to-date views of directories) before hitting the filesystem’s vtable or the block I/O layer at all, and on the fast path[3] taking no locks other than the RCU read lock.

[1] https://www.kernel.org/doc/html/latest/filesystems/path-look...

[2] I was under the impression that it could look up an entire path at once when I wrote my grandparent comment; it seems I was wrong, which on reflection makes sense given you can move directories.

[3] https://www.kernel.org/doc/html/latest/filesystems/path-look...

reply
This is on the mark.

But there's another issue which is what cripples windows for dev! NTFS has a terrible design flaw which is the fact that small files, under 640 bytes, are stored in the MFT. The MFT ends up having serious lock contention so lots of small file changes are slow. This screws up anything Unixy and git horribly.

WSL1 was built on top of that problem which was one of the many reasons it was slow as molasses.

Also why ReFS and "dev drive" exist...

reply
> NTFS has a terrible design flaw which is the fact that small files, under 640 bytes, are stored in the MFT.

Ext4 also stores small (~150B) files inside the inode[1], and so do a number of other filesystems[2]? NTFS was unusually early to the party, but if you’re right that it’s problematic there then something else must also be wrong (perhaps with the locking?) to make it so.

[1] https://www.kernel.org/doc/html/latest/filesystems/ext4/inli...

[2] https://en.wikipedia.org/wiki/Comparison_of_file_systems#All..., the “Inline data” column.

reply