CheatSheet

Control number of GPUs used

Setting: Have a node with multiple GPUs available, but only want to use one.

Set environment variable CUDA_VISIBLE_DEVICES=0.

You can use multiple devices with e.g. CUDA_VISIBLE_DEVICES=1,2.

Note: I used this with a Huggingface example (finetuning on GLUE) and set the environment variables using Azure ML's Environment object:

env = ws.environments['transformers-gpu']
env.environment_variables['CUDA_VISIBLE_DEVICES'] = '0'