OCP_2_6_Group_3

DSSD_26_Composite_Temperature_Variation_Test_Case.py

The test involves the comparison of the Composite Temperature reported on 2 separate drives from the same supplier, model, capacity, FW revision, and under identical workload conditions in the same slot to ensure a fair apples-to-apples comparison.

The objectives of this test case are to verify the following:

  1. Device to device composite temperature variation shall be +/- 1 C under the same environmental conditions, slot location and workload.

  2. Single device.s composite temperature variation shall not vary by more than +/- 1C once it is in steady state under the same environmental condition.

Note that environmental thermal conditions must be controlled by the supplier when testing.

NOTE: The user is required to run this script from the CLI as Yes/No questions will need to be answered

Pass/Fail: Script passes if all tests pass

Open Compute Project Datacenter NVMe SSD Spec v2.6 requirements referenced by this script:

TRPT-1/2/3/4

DSSD_26_Data_Integrity_Test_Case.py

This script will run the following tests:

  1. Issue Identify Controller and verify it’s successful

  2. Check RTD3 Entry Latency field value

  3. Issue a Sequential Write IO for 128kiB, QD=128

  4. Issue Sequential Read IO for 128kiB, QD=128

  5. Execute normal shutdown process and verify CC.SHN and CSTS.SHST values

  6. Execute abrupt shutdown process and verify CC.SHN and CSTS.SHST values

Pass/Fail: Script passes if all tests pass

Open Compute Project Datacenter NVMe SSD Spec v2.6 requirements referenced by this script:

PLP-1

check_performance_logs(dut, perf_metrics, io_bytes, shutdown_type, no_path_timeout)

Checks the performance log using the perf_metrics dict to determine if a “Performance Not Met” warning needs to be issued

Parameters:

  • dut (XML_API) – Device object under test

  • perf_metrics (list | dict) – Performance metrics of Read and Write

  • io_bytes (dict) – dict for read/write bytes

  • shutdown_type (int) – Original Abrupt value

  • no_path_timeout (int) – Original no path timeout value

Return type:

None

collect_performance_logs(dut, io_test, perf_field)

Collects logs from the ADP and prints the performance logs for a given IO command

Parameters:

  • dut (XML_API) – Device object under test

  • io_test (dict) – Custom IO dict

  • perf_field (str | list) – Performance field to be collected. Either “TestReadBytes” | “TestWriteBytes” | “TestLatencyQoS” | “TestWriteIOs” | “TestReadIOs”

Returns:

perf_val_dict – Parsed QoS latency values or dict for Number of read/write bytes

Return type:

dict

execute_IO_Profile(dut, test_type, io_profile, io_runtime=-1, io_pattern=17)
Executes each of the IO Profiles listed in Table 4

  1. Sequential Read (MiB/s) (128kiB, QD = 128)

  2. Sequential Write (MiB/s) (128kiB, QD = 128)

  3. Random Read (KIOPS) (4kiB, QD = 128)

  4. Random Writes (KIOPS) (4kiB, QD = 128)

  5. Random 70R:30W mix (KIOPS) (4kiB, QD = 128)

Parameters:

  • dut (XML_API) – Device object under test

  • test_type (str) – Either ‘Sequential’ | ‘Random’

  • io_profile (str) – IO Profile to be executed

  • io_runtime (int) – Time in seconds for IO to execute

  • io_pattern (int) – Pattern for io to be executed

Return type:

dict

get_runtime_seconds(dut, test_id)

Get the starting or ending timestamp and format it for use in sb_filter

Parameters:

  • dut (XML_API) – Device object under test

  • test_id (str) – I/O testname

Returns:

timestamp – Timestamp formatted in seconds

Return type:

int

restore_orig_abrupt_value(dut, shutdown_type, no_path_timeout)

Restore original value of Abrupt shutdown

Parameters:

  • dut (XML_API) – Device object under test

  • shutdown_type (int) – Original Abrupt value

  • no_path_timeout (int) – Original no path timeout value

Return type:

None

verify_ios(io_tests)

Verifies the succession of an I/O command

Parameters:

io_tests (list) – list of I/O Command dicts

Returns:

io_failed – Returns failed if any io test failed

Return type:

bool

DSSD_26_Device_Panic_Type_10_No_More_NAND_Spares_Read_Only_Mode_Test_Case.py

This script will run the following tests:

  1. Perform sequential write/read for five minutes

  2. Inject Panic Type 10 (No More NAND Spares)

  3. Detect injected Panic Type 10 (No More NAND Spares)

  4. Read log page data

  5. Execute the workflow to attempt recovery

  6. Determine whether the workflow has resolved the injected panic

  7. Verify that error injection is disabled, and device is no longer in an injected panic state

  8. Perform final I/O performance test

Pass/Fail: Script passes if all tests pass

Open Compute Project Datacenter NVMe SSD Spec v2.5 requirements referenced by this script:

CRASH-1/2/3/4

DSSD_26_Device_Panic_Type_1_CPU_Controller_Hang_Test_Case.py

This script will run the following tests:

  1. Perform sequential write/read for five minutes

  2. Inject Panic Type 1 (CPU/Controller Hang)

  3. Detect injected Panic Type 1 (CPU/Controller Hang)

  4. Execute the workflow to attempt recovery

  5. Verify whether the workflow has resolved the injected panic

  6. Verify that error injection is disabled, and device is no longer in an injected panic state

Pass/Fail: Script passes if all tests pass

Open Compute Project Datacenter NVMe SSD Spec v2.5 requirements referenced by this script:

CRASH-1/2/3/4

DSSD_26_Device_Panic_Type_2_NAND_Hang_Test_Case.py

This script will run the following tests:

  1. Perform sequential write/read for five minutes

  2. Inject Panic Type 2 (NAND Hang)

  3. Detect injected Panic Type 2 (NAND Hang)

  4. Execute the workflow to attempt recovery

  5. Determine whether the workflow has resolved the injected panic

  6. Verify that error injection is disabled, and device is no longer in an injected panic state

  7. Perform final I/O test

Pass/Fail: Script passes if all tests pass

Open Compute Project Datacenter NVMe SSD Spec v2.5 requirements referenced by this script:

CRASH-1/2/3/4

DSSD_26_Device_Panic_Type_3_PLP_Defect_Read_Only_Mode_Test_Case.py

This script will run the following tests:

  1. Perform sequential write/read for five minutes

  2. Inject Panic Type 3 (PLP Defect)

  3. Detect injected Panic Type 3 (PLP Defect)

  4. Read log page data

  5. Execute the workflow to attempt recovery

  6. Determine whether the workflow has resolved the injected panic

  7. Verify that error injection is disabled, and device is no longer in an injected panic state

  8. Perform final I/O performance test

Pass/Fail: Script passes if all tests pass

Open Compute Project Datacenter NVMe SSD Spec v2.5 requirements referenced by this script:

CRASH-1/2/3/4

DSSD_26_Device_Panic_Type_4_Logical_Firmware_Error_Test_Case.py

This script will run the following tests:

  1. Perform sequential write/read for five minutes

  2. Inject Panic Type 4 (Logical Firmware Error)

  3. Detect injected Panic Type 4 (Logical Firmware Error)

  4. Execute the workflow to attempt recovery

  5. Determine whether the workflow has resolved the injected panic

  6. Verify that error injection is disabled, and device is no longer in an injected panic state

  7. Perform final I/O performance test

Pass/Fail: Script passes if all tests pass

Open Compute Project Datacenter NVMe SSD Spec v2.5 requirements referenced by this script:

CRASH-1/2/3/4

DSSD_26_Device_Panic_Type_5_DRAM_Corruption_Critical_Path_Test_Case.py

This script will run the following tests:

  1. Perform sequential write/read for five minutes

  2. Inject Panic Type 5 (DRAM Corruption Critical Path)

  3. Detect injected Panic Type 5 (DRAM Corruption Critical Path)

  4. Execute the workflow to attempt recovery

  5. Determine whether the workflow has resolved the injected panic

  6. Verify that error injection is disabled, and device is no longer in an injected panic state

  7. Perform final I/O performance test

Pass/Fail: Script passes if all tests pass

Open Compute Project Datacenter NVMe SSD Spec v2.5 requirements referenced by this script:

CRASH-1/2/3/4

DSSD_26_Device_Panic_Type_6_DRAM_Corruption_Non_Critical_Path_Test_Case.py

This script will run the following tests:

  1. Perform sequential write/read for five minutes

  2. Inject Panic Type 6 (DRAM Corruption Non-Critical Path)

  3. Detect injected Panic Type 6 (DRAM Corruption Non-Critical Path)

  4. Execute the workflow to attempt recovery

  5. Determine whether the workflow has resolved the injected panic

  6. Verify that error injection is disabled, and device is no longer in an injected panic state

  7. Perform final I/O performance test

Pass/Fail: Script passes if all tests pass

Open Compute Project Datacenter NVMe SSD Spec v2.5 requirements referenced by this script:

CRASH-1/2/3/4

DSSD_26_Device_Panic_Type_7_NAND_Corruption_Test_Case.py

This script will run the following tests:

  1. Perform sequential write/read for five minutes

  2. Inject Panic Type 7 (NAND Corruption)

  3. Detect injected Panic Type 7 (NAND Corruption)

  4. Execute the workflow to attempt recovery

  5. Determine whether the workflow has resolved the injected panic

  6. Verify that error injection is disabled, and device is no longer in an injected panic state

  7. Perform final I/O performance test

Pass/Fail: Script passes if all tests pass

Open Compute Project Datacenter NVMe SSD Spec v2.5 requirements referenced by this script:

CRASH-1/2/3/4

DSSD_26_Device_Panic_Type_8_SRAM_Corruption_Test_Case.py

This script will run the following tests:

  1. Perform sequential write/read for five minutes

  2. Inject Panic Type 8 (SRAM Corruption)

  3. Detect injected Panic Type 8 (SRAM Corruption)

  4. Execute the workflow to attempt recovery

  5. Determine whether the workflow has resolved the injected panic

  6. Verify that error injection is disabled, and device is no longer in an injected panic state

  7. Perform final I/O performance test

Pass/Fail: Script passes if all tests pass

Open Compute Project Datacenter NVMe SSD Spec v2.5 requirements referenced by this script:

CRASH-1/2/3/4

DSSD_26_Device_Panic_Type_9_HW_Malfunction_Test_Case.py

This script will run the following tests:

  1. Perform sequential write/read for five minutes

  2. Inject Panic Type 9 (HW Malfunction)

  3. Detect injected Panic Type 9 (HW Malfunction)

  4. Execute the workflow to attempt recovery

  5. Determine whether the workflow has resolved the injected panic

  6. Verify that error injection is disabled, and device is no longer in an injected panic state

  7. Perform final I/O performance test

Pass/Fail: Script passes if all tests pass

Open Compute Project Datacenter NVMe SSD Spec v2.5 requirements referenced by this script:

CRASH-1/2/3/4

DSSD_26_Garbage_Collection_Test_Case.py

The objectives of this test case are as follows:

  1. Verify that the device supports Garbage Collection during periods of no I/O (Idle GC).

  2. Verify that Read latency does not change more than -5% from baseline when the host is issuing Dataset Management commands with AD = 1.

  3. Verify that Read latency does not change more than -5% from baseline after the device has been performing Idle GC.

Open Compute Project Datacenter NVMe SSD Spec v2.6 requirements referenced by this script:

TRIM-5/6/7

collect_performance_logs(test_id)

Collects logs from the ADP and prints the performance logs for a given IO command

compare_qos_performance_logs(test_id, performance_logs, baseline_qos)

Determine if the measured QoS values are within 5% of the baseline QoS values.

precondition(dut, INDIRECT_UNIT_SIZE)

Precondition the DUT by performing 1x128KiB sequential writes and 2x(Indirection Unit Size)KiB random writes.

DSSD_26_IO_Commands_Test_Case.py

This script will run the following tests:

  1. Verify Read/Write/Flush commands are successful

  2. Verify random write I/O performance with FUA=0 and FUA=1 is within 5% of each other

  3. Execute Flush during random write I/O and verify both pass

  4. Do random write/read I/O with FUA=0 for both; then do random write/read I/O with FUA=0 for write and FUA=1 for read, and verify performance is within 5% of each other

  5. If supported, disable volatile write cache, do random write I/O, and verify performance is within 5% of the value in step 2

  6. If supported, enable volatile write cache and verify it was successful

Pass/Fail: Script passes if all tests pass

Open Compute Project Datacenter NVMe SSD Spec v2.5 requirements referenced by this script:

NVMe-IO-1/3/4/5/6/7/8/9/10/11/12/13/14, GETF-2

DSSD_26_IO_Latency_And_Performance_Test_Case.py

This script will run the following tests:

  1. Issue Get-Feature FID=C6h for PLP Health Check Interval data

  2. Issue Set-Feature FID=C6h to set PLP Health Check Interval data for the following values: 05h, 19h, 0Fh

  3. Issue both a Read and Write IO at 128kiB, QD=128, for 5 minutes

  4. Compare IO tests performance to the performance logs in the ADP

Pass/Fail: Script passes if all tests pass

Open Compute Project Datacenter NVMe SSD Spec v2.6 requirements referenced by this script:

PLP-2/7

check_performance_logs(perf_metrics, io_bytes)

Checks the performance log using the perf_metrics dict to determine if a “Performance Not Met” warning needs to be issued

Parameters:

  • perf_metrics (list | dict) – Performance metrics of Read and Write

  • io_bytes (dict) – dict for read/write bytes

Return type:

None

collect_performance_logs(dut, io_test, perf_field)

Collects logs from the ADP and prints the performance logs for a given IO command For a Sequential IO: MiB/s, for a Random IO: IOPS

Parameters:

  • dut (XML_API) – Device object under test

  • io_test (dict) – Custom IO dict

  • perf_field (str | list) – Performance field to be collected. Either “TestReadBytes” | “TestWriteBytes” | “TestLatencyQoS” | “TestWriteIOs” | “TestReadIOs”

Returns:

perf_val_dict – Parsed QoS latency values or dict for Number of read/write bytes

Return type:

dict

execute_IO_Profile(dut, test_type, io_profile, io_runtime=-1, io_pattern=17)
Executes each of the IO Profiles listed in Table 4

  1. Sequential Read (MiB/s) (128kiB, QD = 128)

  2. Sequential Write (MiB/s) (128kiB, QD = 128)

  3. Random Read (KIOPS) (4kiB, QD = 128)

  4. Random Writes (KIOPS) (4kiB, QD = 128)

  5. Random 70R:30W mix (KIOPS) (4kiB, QD = 128)

Parameters:

  • dut (XML_API) – Device object under test

  • test_type (str) – Either ‘Sequential’ | ‘Random’

  • io_profile (str) – IO Profile to be executed

  • io_runtime (int) – Time in seconds for IO to execute

  • io_pattern (int) – Pattern for io to be executed

Return type:

dict

get_runtime_seconds(dut, test_id)

Get the starting or ending timestamp and format in seconds

Parameters:

  • dut (XML_API) – Device object under test

  • test_id (str) – I/O testname

Returns:

timestamp – Timestamp formatted in seconds

Return type:

int

verify_ios(io_tests)

Verifies the succession of an I/O command

Parameters:

io_tests (list) – list of I/O Command dicts

Returns:

io_failed – Returns failed if any io test failed

Return type:

bool

DSSD_26_Precondition_Drive.py

This script will run the following tests:

  1. Precondition the entire drive once with 128KiB sequential writes

  2. Precondition the entire drive twice with INDIRECT_UNIT_SIZE_BYTES random writes

Pass/Fail: Script passes if all tests pass

Open Compute Project Datacenter NVMe SSD Spec v2.6 requirements referenced by this script:

N/A

get_current_queue_depth(dut)

Get the current queue depth on the controller

Parameters:

dut (XML_API) – Device object under test

Returns:

Current queue depth

Return type:

int

set_queue_depth(dut, value)

Set the queue depth on the controller

Parameters:

  • dut (XML_API) – Device object under test

  • value (int) – Queue depth to set

Return type:

None

DSSD_26_Temperature_Sensors_And_Composite_Temperature_Calculation_Test_Case.py

The objectives of this test case are to verify the following:

  1. That the device supports the number of sensors indicated by the vendor parameter data.

  2. That the equation, settings, thresholds and artifacts provided by the vendor, and the raw sensor data, can be used to calculate a composite temperature value that matches the value reported in the SMART Log data.

NOTE: The user is required to run this script from the CLI as Yes/No questions will need to be answered

Pass/Fail: Script passes if all tests pass

Open Compute Project Datacenter NVMe SSD Spec v2.6 requirements referenced by this script:

TRPT-1 to TRPT-4