count
  • Supermicro SYS-7048GR-TR
  • (2) GTX 1080 FE
  • (2) GTX 1070 FE

Install the OS

Version:

  • Ubuntu 16.04.3-server-amd64

Install Options:

  • Standard System utilities
  • OpenSSH Server

Once the install has completed SSH into the server and update the OS to the latest packages:

$ sudo dpkg --remove-architecture i386
$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get dist-upgrade
$ sudo reboot

When the server finishes rebooting, SSH back in and install required packages:

$ sudo apt-get install xserver-xorg-core xdm xserver-xorg-dev libgtk-3-0 libxnvctrl-dev p7zip-full pkg-config gcc --no-install-recommends

Stop the X display manager:

$ sudo systemctl stop xdm

Install NVidia Drivers

Check for the latest driver at http://www.nvidia.com/Download/Find.aspx, the link below was valid as of Nov 11, 2017.

$ wget http://us.download.nvidia.com/XFree86/Linux-x86_64/387.22/NVIDIA-Linux-x86_64-387.22.run

Make the installer executable and launch:

$ chmod +x NVIDIA-Linux-x86_64-387.22.run && sudo ./NVIDIA-Linux-x86_64-387.22.run
  • Select Accept license
  • Select OK to warning about 32-bit libraries
  • Select NO to generate Xorg
  • Select OK to complete

Generate an Xorg config that enables GPU fan control:

$ sudo nvidia-xconfig -s -a --force-generate --allow-empty-initial-configuration \
--cool-bits=12 --registry-dwords="PerfLevelSrc=0x2222" --no-sli \
--connected-monitor="DFP-0" -o /etc/X11/xorg.conf

Add Xorg configuration options to disable the NVidia driver watchdog.

$ sudo sh -c "sed -i '/^[[:blank:]]*Driver[[:blank:]]*\"nvidia/a\ \ \ \ Option         \"Interactive\" \"False\"' /etc/X11/xorg.conf"

Add settings required for Hashcat to control the GPU fans from terminal sessions (e.g SSH)

$ sudo sh -c "cat >> /etc/X11/xdm/Xsetup << EOF
export PATH=/bin:/usr/bin:/sbin
export HOME=/root
export DISPLAY=:0
xset -dpms
xset s off
xhost +
EOF"

Set the DISPLAY environment variable during login:

$ echo "export DISPLAY=:0" >> ~/.bashrc

Now reboot to verify all changes:

$ sudo reboot

When the reboot completes SSH back in and verify everything works:

$ nvidia-smi --query-gpu=gpu_name --format=csv,noheader

GeForce GTX 1080
GeForce GTX 1070
GeForce GTX 1080
GeForce GTX 1070

$ nvidia-settings -tq GPUCurrentFanSpeed

27
28
27
28

Install Hashcat

Download and unpack:

$ wget https://hashcat.net/files/hashcat-4.0.1.7z
$ 7z x hashcat-4.0.1.7z
$ cd hashcat-4.0.1/

Verify that Hashcat recognizes the video cards:

$ ./hashcat64.bin -I

hashcat (v4.0.1) starting...

OpenCL Info:

Platform ID #1
  Vendor  : NVIDIA Corporation
  Name    : NVIDIA CUDA
  Version : OpenCL 1.2 CUDA 9.1.0

  Device ID #1
    Type           : GPU
    Vendor ID      : 32
    Vendor         : NVIDIA Corporation
    Name           : GeForce GTX 1080
    Version        : OpenCL 1.2 CUDA
    Processor(s)   : 20
    Clock          : 1733
    Memory         : 2028/8113 MB allocatable
    OpenCL Version : OpenCL C 1.2
    Driver Version : 387.12
    <SNIP>

Run a simple benchmark to verify Hashcat is operational:

$ ./hashcat64.bin -m1000 -b

OpenCL Platform #1: NVIDIA Corporation
======================================
* Device #1: GeForce GTX 1080, 2028/8113 MB allocatable, 20MCU
* Device #2: GeForce GTX 1070, 2028/8113 MB allocatable, 15MCU
* Device #3: GeForce GTX 1080, 2028/8113 MB allocatable, 20MCU
* Device #4: GeForce GTX 1070, 2028/8113 MB allocatable, 15MCU

Hashtype: NTLM

Speed.Dev.#1.....: 41352.5 MH/s (64.86ms)
Speed.Dev.#2.....: 30969.0 MH/s (64.95ms)
Speed.Dev.#3.....: 41691.6 MH/s (64.33ms)
Speed.Dev.#4.....: 31089.5 MH/s (64.70ms)
Speed.Dev.#*.....:   145.1 GH/s

Test GPU fan control by Hashcat (fans should show at 100% during the test):

$ echo -n "foobar" | md5sum | awk '{print $1}' > hash.txt
$ ./hashcat64.bin -m0 -a0 --gpu-temp-retain=10  hash.txt

<SNIP>
HWMon.Dev.#1.....: Temp: 47c Fan:100% Util:  0% Core:1607MHz Mem:4513MHz Bus:16
HWMon.Dev.#2.....: Temp: 49c Fan:100% Util:  0% Core:1506MHz Mem:3802MHz Bus:16
HWMon.Dev.#3.....: Temp: 48c Fan:100% Util:  0% Core:1607MHz Mem:4513MHz Bus:16
HWMon.Dev.#4.....: Temp: 50c Fan:100% Util:  0% Core:1506MHz Mem:3802MHz Bus:16

(Optional) Install John the Ripper

$ sudo apt-get install libssl-dev librexgen
$ git clone -b bleeding-jumbo --single-branch https://github.com/magnumripper/JohnTheRipper.git
$ cd JohnTheRipper/src
$ ./configure
$

(Optional) Install OpenCL runtime for CPU cracking

$ sudo apt-get install clinfo ocl-icd-libopencl1 lsb-core --no-install-recommends

OpenCL™ Runtime for Intel® Core™ and Intel® Xeon® Processors

Check for the latest driver at https://software.intel.com/en-us/articles/opencl-drivers, the link below was valid as of Nov 14, 2017.

$ wget http://registrationcenter-download.intel.com/akdlm/irc_nas/9019/opencl_runtime_16.1.1_x64_ubuntu_6.4.0.25.tgz

Unpack and run the installer:

$ tar zxf opencl_runtime_16.1.1_x64_ubuntu_6.4.0.25.tgz
$ cd opencl_runtime_16.1.1_x64_ubuntu_6.4.0.25/
$ sudo ./install.sh

Comments