Repository navigation
Write sparse files without allocating their gaps, so multi-GB file tests pass - #54
Conversation
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 5 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
There was a problem hiding this comment.
This PR fixes sparse file handling by switching to host-backed storage before allocating large gaps in memory. It adds append-mode semantics (writes always go to end), allows read-only directory channels, and ensures large files on read-only mounts use host handles. Eleven previously failing OpenJDK tests now pass with peak RSS under 500 MB.
review-bot, model nvidia/nemotron-3-ultra-550b-a55b:free, verdict approve
There was a problem hiding this comment.
The PR fixes sparse file handling by switching to host handles before allocating gap memory, adds append-mode semantics matching O_APPEND, and supports read-only directory channels. Eleven previously failing OpenJDK tests now pass. The changes span VFS, native I/O, NIO, and tests.
review-bot, model nvidia/nemotron-3-ultra-550b-a55b:free, verdict approve
There was a problem hiding this comment.
The PR fixes sparse file handling by switching to host handles before allocating gap memory (preventing OOM on multi-GB writes), adds O_APPEND semantics for append-mode descriptors, allows read-only directory channel opens (failing reads with 'Is a directory'), and makes large files on read-only mounts use host handles. Eleven previously failing OpenJDK tests now pass with peak RSS 258-497 MB.
Not approved: 1 earlier point is still open in the threads.
review-bot, model nvidia/nemotron-3-ultra-550b-a55b:free, verdict comment
There was a problem hiding this comment.
The PR fixes sparse file handling so that writes past 4 MiB switch to host-based storage, preventing memory allocation for gaps. This enables eleven OpenJDK multi-GB file tests to pass. Changes include modifying the write method to stream to host before allocating gaps, adding DirectoryFile class for read-only directory support, updating canOpenFile and openFile for proper writability checks, and adding append mode and mapped buffer write-back support.
Not approved: 1 earlier point is still open in the threads.
review-bot, model nvidia/nemotron-3.5-lightning:free, verdict comment
Eleven OpenJDK tests that write multi-GB files now pass instead of running out of memory:
FileChannel/Transfer2GPlus,Transfer4GBFile,TransferTo6GBFile,LoopingTruncate,Channels/TransferTo_2GB_transferTo,TransferTo_2GB_transferFrom,Files/SBC,FileInputStream/LargeFileAvailable,ZipFile/CenSizeTooLarge,ZipFile/EndOfCenValidationandzipfs/EndOfCenValidation.All of them write after a large seek, the way sparse files are made. #47 moved a file to the host once it grew past 4 MiB, but only after the write: a write at 2 GB first allocated the whole 2 GB gap in memory. Fixes:
O_APPENDdoes. Positional writes and mapped-buffer write-back keep their position. The transfer tests found this once the allocation was fixed.SBCdoes this); reading it fails withIs a directory, and a snapshot is refused while one is open.Tests:
test/java/StreamingFile.javaadds a write far past the end and appends after a seek, matching HotSpot.test/nio.test.tscovers the append and directory cases.passing.txtwith their durations intimes.tsv.pnpm typecheckandpnpm testpass.Noticed, not fixed: