一、电脑
nvidia 的系统管理界面 (nvidia-smi)，可以收集各种级别的信息，查看显存使用情况。此外, 可以启用和禁用 GPU 配置选项 (如 ECC 内存功能)
(pytorch1.2) C:\Users\Administrator>nvidia-smi -q
==============NVSMI LOG==============
Timestamp : Mon Oct 31 20:21:25 2022
Driver Version : 473.34
CUDA Version : 11.4
Attached GPUs : 1
GPU 00000000:01:00.0
Product Name : NVIDIA GeForce GT 730%%显卡名字及型号
Product Brand : GeForce
Display Mode : N/A
Display Active : N/A
Persistence Mode : N/A
MIG Mode
Current : N/A
Pending : N/A
Accounting Mode : N/A
Accounting Mode Buffer Size : N/A
Driver Model
Current : WDDM
Pending : WDDM
Serial Number : N/A
GPU UUID : GPU-0c294081-5ddc-5293-f5a1-f504f7234b33
Minor Number : N/A
VBIOS Version : 80.28.c4.00.28
MultiGPU Board : N/A
Board ID : N/A
GPU Part Number : N/A
Module ID : 0
Inforom Version
Image Version : N/A
OEM Object : N/A
ECC Object : N/A
Power Management Object : N/A
%%底下还有很多省略
(pytorch1.2) C:\Users\Administrator>nvidia-smi
Mon Oct 31 20:15:15 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 473.34 Driver Version: 473.34 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... WDDM | 00000000:01:00.0 N/A | N/A |
| 23% 0C P8 N/A / N/A | 757MiB / 2048MiB | N/A Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
参数显示表格如下
| Fan | 风扇转速（0%–100%），N/A表示没有风扇 |
|---|---|
| Temp | GPU温度（GPU温度过高会导致GPU频率下降） |
| Perf | 性能状态，从P0（最大性能）到P12（最小性能） |
| Pwr | GPU功耗 |
| Persistence-M | 持续模式的状态（持续模式耗能大，但在新的GPU应用启动时花费时间更少） |
| Bus-Id | GPU总线，domain🚌device.function |
| Disp.A | Display Active，表示GPU的显示是否初始化 |
| Memory-Usage | 显存使用率 |
| Volatile GPU-Util | GPU使用率 |
| ECC | 是否开启错误检查和纠正技术，0/DISABLED, 1/ENABLED |
| Compute M. | 计算模式，0/DEFAULT,1/EXCLUSIVE_PROCESS,2/PROHIBITED |
还不太清楚补充其他图片标记
二、服务器中
(base) wlc@hzsfxy-SYS-4028GR-TR2:~$ nvidia-smi
Mon Oct 31 20:25:16 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 495.44 Driver Version: 495.44 CUDA Version: 11.5 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA A100-PCI... Off | 00000000:06:00.0 Off | 0 |
| N/A 65C P0 80W / 250W | 5414MiB / 40536MiB | 100% Default |
| | | Disabled |
+-------------------------------+----------------------+----------------------+
| 1 NVIDIA A100-PCI... Off | 00000000:07:00.0 Off | 0 |
| N/A 42C P0 39W / 250W | 430MiB / 40536MiB | 0% Default |
| | | Disabled |
+-------------------------------+----------------------+----------------------+
| 2 NVIDIA A100-PCI... Off | 00000000:82:00.0 Off | 0 |
| N/A 43C P0 35W / 250W | 430MiB / 40536MiB | 0% Default |
| | | Disabled |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 2181 G /usr/lib/xorg/Xorg 4MiB |
| 0 N/A N/A 10819 G /usr/lib/xorg/Xorg 4MiB |
| 0 N/A N/A 1415253 G /usr/lib/xorg/Xorg 4MiB |
| 0 N/A N/A 1561870 C ./shell-xorg 5373MiB |
| 1 N/A N/A 2181 G /usr/lib/xorg/Xorg 4MiB |
| 1 N/A N/A 10819 G /usr/lib/xorg/Xorg 4MiB |
| 1 N/A N/A 1415253 G /usr/lib/xorg/Xorg 4MiB |
| 1 N/A N/A 1561870 C ./shell-xorg 413MiB |
| 2 N/A N/A 2181 G /usr/lib/xorg/Xorg 4MiB |
| 2 N/A N/A 10819 G /usr/lib/xorg/Xorg 4MiB |
| 2 N/A N/A 1415253 G /usr/lib/xorg/Xorg 4MiB |
| 2 N/A N/A 1561870 C ./shell-xorg 413MiB |
+-----------------------------------------------------------------------------+
(base) wlc@hzsfxy-SYS-4028GR-TR2:~$ nvidia-smi -q
==============NVSMI LOG==============
Timestamp : Mon Oct 31 20:29:07 2022
Driver Version : 495.44
CUDA Version : 11.5
Attached GPUs : 3
GPU 00000000:06:00.0
Product Name : NVIDIA A100-PCIE-40GB
Product Brand : NVIDIA
Product Architecture : Ampere
Display Mode : Enabled
Display Active : Disabled
Persistence Mode : Disabled
MIG Mode
Current : Disabled
Pending : Disabled
Accounting Mode : Disabled
Accounting Mode Buffer Size : 4000
Driver Model
Current : N/A
Pending : N/A
Serial Number : 1560821008125
GPU UUID : GPU-d2195b63-e500-21e5-0584-842d59a69b8d
Minor Number : 0
VBIOS Version : 92.00.25.00.08
MultiGPU Board : No
Board ID : 0x600
GPU Part Number : 900-21001-0100-030
Module ID : 0
Inforom Version
Image Version : 1001.0200.00.04
OEM Object : 2.0
ECC Object : 6.16
Power Management Object : N/A
GPU Operation Mode
Current : N/A
Pending : N/A
GSP Firmware Version : N/A
GPU Virtualization Mode
Virtualization Mode : None
Host VGPU Mode : N/A
IBMNPU
Relaxed Ordering Mode : N/A
PCI
Bus : 0x06
Device : 0x00
Domain : 0x0000
Device Id : 0x20F110DE
Bus Id : 00000000:06:00.0
Sub System Id : 0x145F10DE
GPU Link Info
PCIe Generation
Max : 3
Current : 3
Link Width
Max : 16x
Current : 16x
Bridge Chip
Type : N/A
Firmware : N/A
Replays Since Reset : 0
Replay Number Rollovers : 0
Tx Throughput : 0 KB/s
Rx Throughput : 4000 KB/s
Fan Speed : N/A
Performance State : P0
Clocks Throttle Reasons
Idle : Not Active
Applications Clocks Setting : Not Active
SW Power Cap : Not Active
HW Slowdown : Not Active
HW Thermal Slowdown : Not Active
HW Power Brake Slowdown : Not Active
Sync Boost : Not Active
SW Thermal Slowdown : Not Active
Display Clock Setting : Not Active
FB Memory Usage
Total : 40536 MiB
Used : 5070 MiB
Free : 35466 MiB
BAR1 Memory Usage
Total : 65536 MiB
Used : 59 MiB
Free : 65477 MiB
Compute Mode : Default
Utilization
Gpu : 100 %
Memory : 3 %
Encoder : 0 %
Decoder : 0 %
Encoder Stats
Active Sessions : 0
Average FPS : 0
Average Latency : 0
FBC Stats
Active Sessions : 0
Average FPS : 0
Average Latency : 0
Ecc Mode
Current : Enabled
Pending : Enabled
ECC Errors
%%底下省略
