Streamlining System Reliability with Automated Burn-In Tests Using Phoronix Test Suite

by Pete
8 minutes read

Ensuring the reliability and stability of computer systems is paramount, especially when preparing for intensive workloads or before deploying in a server environment. Burn-in testing, which pushes hardware components to operate at high levels of performance for extended periods, is a crucial method for identifying potential failures early.

This guide will explore how to automate comprehensive burn-in tests using the Phoronix Test Suite, a powerful tool designed for Linux systems.

Warning

These tests can damage your system components if you don’t have adequate cooling and the tests could highlight underlying problems with your hardware and system.

Please keep an eye on your system temperatures and stop the tests if you’re experiencing system instability or faults.

***Please use are your own risk. LinuxMO is not responsible for any damage incurred by running these tests and scripts.***

Why Opt for Burn-In Testing?

Burn-in testing is essential for pushing hardware to its limits, ensuring components can withstand real-world pressures. Here’s why it’s crucial:

  • Detects Defective Hardware Early: Catch hardware faults before systems go live.
  • Assures Stability Under Load: Confirms systems can handle intense operations without buckling.
  • Extends Hardware Lifespan: Verifies component durability and operational reliability.

Preparing for Testing

Start by installing the Phoronix Test Suite. It’s available through most Linux distribution repositories and can be easily installed using package managers like apt or yum. Alternatively, download it from the Phoronix Test Suite website:

sudo apt-get install phoronix-test-suite

Setting Up Automated Tests

Automation is key for efficiently running multiple tests. We’ll use an XML file to specify which tests to run and a bash script to execute them without manual input.

1
Configuring Batch Mode – Important!

First, configure the batch mode settings to ensure the test suite runs without manual intervention:

sudo phoronix-test-suite batch-setup

During setup, you’ll encounter prompts like these:

  • Save test results when in batch mode (Y/n): Choose ‘n’ if you do not need to keep a record within Phoronix’s own system, perhaps because you’re logging results another way.
  • Run all test options (Y/n): Select ‘y’ to automatically run all available test options for any given benchmark, which is useful for comprehensive testing.

These settings help the test suite to operate autonomously, ideal for scenarios like overnight testing or integration into continuous deployment pipelines.

2
Script for Running the Batch Tests

Create a bash script to automate the execution of the Phoronix Test Suite using the batch file. In this instance, I called mine run_phoronix_tests.sh

nano run_phoronix_tests.sh 

Update the RESULTS_DIR with your desired folder.

#!/bin/bash
# Important Setup
# make sure you run 
# sudo phoronix-test-suite batch-setup 
# to configure batch mode
# Define directory to store results
RESULT_DIR="/home/pfarmer/Downloads/_No Sync"
mkdir -p "$RESULT_DIR"
# Enable batch mode for Phoronix Test Suite (may be redundant with XML, but included for clarity)
export PTS_BATCH_MODE=1
# Function to prompt for skipping a test
prompt_skip_test() {
    read -t 10 -n 1 -p "Press any key within 10 seconds to skip the $1 test or wait to continue... " key
    echo # Move to a new line
    if [[ $key ]]; then
        echo "Skipping $1 test..."
        return 1 # Return 1 to indicate skip
    else
        echo "Running $1 test..."
        return 0 # Return 0 to indicate no skip
    fi
}
# Function to run a test and save the results
run_test() {
    TEST_ID=$1
    TEST_NAME=$2
    # Call skip prompt
    prompt_skip_test "$TEST_NAME"
    if [[ $? -eq 1 ]]; then
        echo "$TEST_NAME skipped." &> "$RESULT_DIR/$TEST_NAME.txt"
        return # Skip the test execution
    fi
    echo "Executing $TEST_NAME..."
    # Use user-config-file option to specify the XML configuration
    sudo sudo phoronix-test-suite batch-benchmark $TEST_ID &> "$RESULT_DIR/$TEST_NAME.txt"
    echo "$TEST_NAME completed! Results saved to $RESULT_DIR/$TEST_NAME.txt"
}
# List of tests to run, corresponding to the blog article
run_test "pts/cpuminer-opt" "CPU_Stress_Test"
run_test "pts/ramspeed" "Memory_Stability_Test"
run_test "pts/unigine-valley" "GPU_Endurance_Test"
run_test "pts/aio-stress" "Motherboard_IO_Capability_Test"
run_test "pts/fio" "Hard_Drive_Stress_Test"
run_test "pts/sysbench" "Overall_System_Stability"
echo "All tests completed! Check $RESULT_DIR for detailed results."

If you’d like fewer or more tests, please add the relative command to a new line using the same format shown in the script.

Additionally, I found that sysbench will ask for what tests and the script hangs waiting for input. So, I added in a test using expect command to substitute inputs for me. Seems to work OK. And I added the ability to skip tests if you want to get to a certain test quickly.

3
Make the script executable

sudo chmod +x run_phoronix_tests.sh

4
Then execute the script

sudo ./run_phoronix_tests.sh

5
Key Notes

  1. Permissions: Ensure the script has the appropriate permissions to run. It’s common to require sudo for installing necessary components or handling system-level tests.
  2. Batch Mode: This script assumes that you have configured your Phoronix Test Suite with batch-setup appropriately, as discussed earlier.
  3. Result Handling: The script redirects output to a file for each test, which can be useful for reviewing results later without having to parse through a unified log.
  4. Adding Tests: If you want to add more tests or change existing ones, modify the run_test function calls with the appropriate test identifiers.

Conclusion

Automating burn-in tests using the Phoronix Test Suite significantly enhances the stability and durability of Linux systems. By setting up batch mode and using XML files for test configurations, you can ensure comprehensive testing without manual intervention.

This approach not only saves time but also provides a deeper insight into system performance under stress. Start your automated testing today and ensure your hardware is robust and reliable.