finalizers for model objects

[npcooley]

When i remove objects in R, i.e. rm(my_model), i don't see them removed from device memory. Are associated finalizers for this process or is restarting the R session the only way to clear the device memory?

[dfalbel]

They should eventually get removed when the R garbage collector executes. you can force a gc() too. But more importantly, if torch needs to allocate more memory on the gpu device, it will free them too.

YOu can read more in https://torch.mlverse.org/docs/articles/memory-management

[npcooley]

I'll try and get a more reproducible example here this weekend, but running torch in an rstudio server instance, calling gc() does not clear processes from the device. i.e. i will still see active processes like this post object removal:

> rm(list = ls())
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3756193 200.7    7028200 375.4  5292220 282.7
Vcells 5585821  42.7   11434467  87.3 11827559  90.3
> Sys.sleep(30)
> system("nvidia-smi")
Fri Jul 17 11:06:18 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.105.08             Driver Version: 580.105.08     CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA A30                     Off |   00000000:CA:00.0 Off |                    0 |
| N/A   28C    P0             30W /  165W |     383MiB /  24576MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A          312827      C   ...b/rstudio-server/bin/rsession        374MiB |
+-----------------------------------------------------------------------------------------+

[dfalbel]

libtorch doesn't always give back memory to the OS right after the deletion. this is because it uses a cache allocator. So even if memory is no longer used by the session, the OS can still report memory usage.

In the article I linked, you can read more:

CUDA memory tends to be scarcer than CPU memory, also, allocation must be faster otherwise allocation overhead can counterbalance the speed up of GPU. To make allocations very fast and to avoid segmentation, LibTorch uses a caching allocator to manage the GPU memory, ie. once LibTorch allocated CUDA memory it won’t give it back to the operation system, instead it reuses that memory for future allocations. This means that nvidia-smi or nvtop will not report the amount of memory used by tensors, but the memory LibTorch has reserved from the OS. You can use torch::cuda_memory_summary() to query exactly the memory used by LibTorch.