[FEA]: Settable memory pool attributes and trim() for pool-backed memory resources #3261

Is this a duplicate?

• I confirmed there appear to be no duplicate issues for this request and that I agree to the Code of Conduct

Area

cuda.core

Is your feature request related to a problem? Please describe.

DeviceMemoryResource, PinnedMemoryResource and ManagedMemoryResource expose attributes, but every field is read-only, and there is no way to trim a pool.

• Measuring the peak memory of each phase of a computation needs used_mem_high and its documented reset (setting it to 0). Stream-ordered allocations don't go through CuPy's memory hooks, so the pool's watermark is the only source.
• DeviceMemoryResource(device) without options raises the default pool's release_threshold to ULLONG_MAX, for every user of that pool in the process. A CUDA-accelerated simulation library I am developing, cuNIBS allocates temporaries with cudaMallocAsync from the default pool and relies on the pool returning unused memory at the next synchronization (_memory.py). cuda.core offers no way to set the threshold back.

Describe the solution you'd like

• release_threshold and the three reuse_* policies become settable.
• reserved_mem_high and used_mem_high accept 0 to reset the watermark, like GraphMemoryResourceAttributes.
• trim(min_bytes_to_keep=0) wraps cuMemPoolTrimTo, like GraphMemoryResource.trim().
• Pool attribute calls and trim() run in relaxed capture mode. This also fixes the existing getters: on main they fail with CUDA_ERROR_STREAM_CAPTURE_UNSUPPORTED and invalidate a capture in progress in global mode on any thread, or in thread-local mode on the calling thread. cuda.core: fix pool setup and builder teardown under stream capture #2838 fixed the same problem for MP_raise_release_threshold (cuda.core: DeviceMemoryResource setup is refused under non-relaxed stream capture, and tearing down the invalidated GraphBuilder segfaults #2834).

I opened #3236 with this change before opening an issue. I'll revise or close it depending on the outcome here.

Describe alternatives you've considered

Stop raising the default pool's release_threshold in DeviceMemoryResource(device). That would undo #771, which raised it because allocations from the default pool were slow with a threshold of 0.

Additional context

Open question: should setters work on a resource that wraps the device's default pool? A change there affects every user of that pool in the process. cuda.core already changes that pool's release_threshold implicitly. #3236 allows setters on it and documents the effect. Allowing setters only on owned pools would rule out the release_threshold case above. #2362 raises the same question for max_pool_size and process-global pool changes.