Running CUDA Profilers on Linux: GPU Optimizations

Author: Greg Gutmann
Affiliation: Tokyo Institute of Technology, Nvidia University Ambassador, Nvidia DLI

Introduction

In this post, I will outline what is needed to install and run several of Nvidia’s profiling tools.

WARNING: This was written on September 15th, 2020. The versions and processes involved may change over time. I am using CUDA 11.0, but CUDA 11.1 has already been released.

Nvidia Visual Profiler: nvvp

After installing CUDA version 11.0, nvvp is located in:

/usr/local/cuda-11.0/bin/

You must install a Java Runtime Environment (JRE) 1.8 to use nvvp as mentioned in [Nvidia’s documentation]. I used [OpenJDK].

sudo apt-get install openjdk-8-jre

Then to run the nvvp, use the following command, assuming you are also using the same OpenJDK.

nvvp -vm /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java

Nsight Systems: nsight-sys

If using CUDA 11.0 Nsight systems should already be installed in /usr/local/cuda-11.0/bin/ as nsight-sys.

If it is already installed, you can skip the next subsection and go to “Running Nsight Systems.”

Process for Installing or Getting a Newer Version

By following the first link below, line 2, you should find something similar to the following image, then click the “Download Now” button.

// Nsight-systems page
https://developer.nvidia.com/nsight-systems
// Nsight Systems Documentation
https://docs.nvidia.com/nsight-systems/InstallationGuide/index.html
Nsight Systems page: Click Download Now

When I went through this process a few days ago, the newest version was 2020.3, whereas now 2020.4 is available, as shown in the image below. So with frequent updates, you will need to remember your paths will be slightly different than the ones in this post.

I chose the linux host .run installer.

Nvidia’s Download Center for Nsight Systems

After downloading, you will need to change permissions on the .run file; keep in mind the version number. At the time, I ran the following line below in a terminal window.

sudo chmod +x NVIDIA_Nsight_Systems_Linux_2020.3.1.72.run
sudo ./NVIDIA_Nsight_Systems_Linux_2020.3.1.72.run

You can use the space bar to scroll faster for license agreement.

Then during the installation process, some of the important things to watch out for are lines such as the following.

install path: [ default is /home/one/nsight-systems-2020.3.1 ]
To uninstall the Nsight Systems 2020.3.1, please delete "/home/one/nsight-systems-2020.3.1"

Running Nsight Systems

Nsight Systems is located at the following locations for me since it came with CUDA 11.0 and I installed it after the fact as well:

/usr/local/cuda-11.0/bin/
/home/greg/nsight-systems-2020.3.1/bin

You can run by typing nsight-sys in a terminal from any directory, but you may encounter permission issues.

So instead, I ran the following to run Nsight Systems with elevated permissions:

sudo /home/greg/nsight-systems-2020.3.1/bin/nsight-sys

[Nvidia Profiling Documentation]

Possible Warnings When Profiling

I received several warnings that some metrics were not collected due to the “perf paranoid level.” To work around this, I ran the following command, but be cautious when doing so as you may be making your system more vulnerable:

sudo sh -c 'echo 1 >/proc/sys/kernel/perf_event_paranoid'

A post by Nvidia which covers this a bit: https://developer.nvidia.com/blog/nvidia-nsight-systems-containers-cloud/

Nsight Compute: ncu-ui

If using CUDA 11.0 Nsight compute should already be installed in /usr/local/cuda-11.0/bin/ as ncu-ui.

If it is already installed, you can skip the next subsection and go to “Running Nsight Compute.”

Process for Installing or Getting a Newer Version

Nsight Compute can be found at the following link; it has also been updated in the past few days, from version 2020.1.2 to 2020.2.0. Click the green download button, then choose the Linux version.

https://developer.nvidia.com/nsight-compute
NVIDIA Nsight Compute Page
Nvidia’s Download Center for Nsight Compute

Then I ran the following commands after downloading to change permissions on the run file and then run the installer. Watch for a difference in versions.

sudo chmod +x nsight-compute-linux-2020.1.2.4-28820667.run
sudo ./nsight-compute-linux-2020.1.2.4-28820667.run

You can use the space bar to scroll faster for license agreement.

Then during the installation process, some of the important things to watch out for are lines such as the following.

Enter install path: [ default is /usr/local/NVIDIA-Nsight-Compute-2020.1 ]

// Optional to say yes or no
Would you like a symlink created at /usr/local/NVIDIA-Nsight-Compute? Existing folder will be deleted. (YES/NO): YES

Location /usr/local/NVIDIA-Nsight-Compute/

Running Nsight Compute

Nsight Compute is located at the following locations for me since it came with CUDA 11.0, I installed it after the fact, and I added a symlink:

/usr/local/cuda-11.0/bin/
/usr/local/NVIDIA-Nsight-Compute-2020.1/
/usr/local/NVIDIA-Nsight-Compute/

You can run by typing ncu-ui in a terminal from any directory, but you may encounter permission issues.

I ran Nsight Compute with the following command:

sudo /usr/local/NVIDIA-Nsight-Compute/ncu-ui

Contact me if you would like to use the contents of this post. Thanks.
Copyright © 2020 by Gregory Gutmann

Leave a Reply

Close Menu