SBExpress NVMe sb_i2c2 User Guide

Version 10.8 — April 2025



Contents


Overview

Many of the subsystems on SANBlaze systems use the I2C bus to set, check, and clear settings on the main boards, risers, and adapters. The sb_i2c program provides access to the various I2C devices on the system. Starting with the Gen5 systems (SBExpress-RM5 and -DT5), the sb_i2c program has been replaced by sb_i2c2, which is more comprehensive, faster, and contains help specific to each use case on a per riser or adapter (m.2) basis rather than system wide. The sb_i2c program should be considered deprecated and sb_i2c2 should be used in all cases. The original sb_i2c command simply calls through to sb_i2c2 and remains in place for backward compatibility purposes only.


Glitch, Clock, and Power Commands

For the SBExpress-DT, SBExpress-RM and SBExpress-RM5, use the syntax in the examples shown below for Glitch, Clock, and Power.

1. Glitch PCIe Reset signal (HP_PERST_) at slot 0 for 1 Second:

sb_i2c -d 0 -f HP_PERST_ -w 0 -g 1000000

2. Disable PCIe Clock signal (HP_CLKEN_) at slot 0 for 1 Second:

sb_i2c -d 0 -f HP_CLKEN_ -w 1 -g 1000000

3. Disable POWER (HP_PWREN) at slot 0 for 1 Second:

sb_i2c -d 0 -f HP_PWREN -w 0 -g 1000000

4. Turn power off/on (HP_PWREN) at slot 0 using the Hot Plug Controller:

sb_i2c -d 0 -f HP_PWREN -w 0|1

5. Control power without PERST. The feature PERST_12V_L and alias perst_power bracket the power on with PERST. For example:

sb_i2c -d 0 -f power -w 0
INFO: 00[0x61] 0x05[05:04] DISABLE_12V_L Def=0x01 Cur=0x00

sb_i2c -d 0 -f power -w 1
INFO: 00[0x61] 0x05[05:04] DISABLE_12V_L Def=0x01 Cur=0x01

sb_i2c -d 0 -f perst_power -w 0
INFO: 00[0x61] 0x05[05:04] DISABLE_12V_L Def=0x01 Cur=0x00

sb_i2c -d 0 -f perst_power -w 1
INFO: 00[0x60] 0x05[05:04] PORT0_PERST_L Def=0x01 Cur=0x00
INFO: 00[0x61] 0x05[05:04] DISABLE_12V_L Def=0x01 Cur=0x01
INFO: 00[0x60] 0x05[05:04] PORT0_PERST_L Def=0x01 Cur=0x01

Power Supply

Read Power

SBExpress-RM standard power supply (uppercase -M):

sb_i2c -n 1 -M
Measuring power from power supply at 0x59 reg 0x8d
INFO: Power supply temperature = 42.8750
INFO: Power supply power = 87.0W
INFO: Power supply voltage = 12.2793V
INFO: Power supply current = 7.1250A

SBExpress-RMI industrial power supply (lowercase -m):

sb_i2c -n 1 -m
INFO: System 1 SBExpress-RM SN=920A8110006 Rev=R03 i2c=/dev/i2c-0
      MI_i2c=/dev/i2c-1 VLUN_Port=0
INFO: restdir = /virtualun/webs/web/rest/sanblazes/1/sn
INFO: 01[-3] load12V=12378mV, max12V=14498mV, min12V=9653mV

Voltage

Set 12V rail to X power +/- 10% of 12 Volts:

sb_i2c -n 1 -v 12000

-v N — Set the voltage at the power supply to N mV. Use -v 0 to recalibrate the power supply (max/min/nominal).

Caution: Do not initiate a calibration with drives populated. The power supply will be set to Min and Max and may adversely affect devices under test.


FANS

  • Six variable speed fans are controlled by sb_fans.service

  • Fans are controlled through two zones – temperature and speed – via a configuration file called /etc/sb_fans.conf

Checking the Status of FANs

To check the status of the fans, you can run the commands shown after typing:

cat /rest/sanblazes/1/fan

Example output:

fan1fail  fan1temp  fan2rpm  fan3fail  fan3temp  fan4rpm  fan5fail
fan5temp  fan6rpm
fan1rpm   fan2fail  fan2temp fan3rpm   fan4fail  fan4temp fan5rpm
fan6fail  fan6temp

These commands check the fan temperature, RPM, and fan failure (indicates the fan isn’t running). Note that the “1” in the example above indicates there is one SBExpress-RM device (you can run up to two chassis daisy chained together).

Use the individual commands to check for temperature, RPM, or fan failure:

[vlun@vlun-111 ~]$ cat /rest/sanblazes/1/fan1temp
26
[vlun@vlun-111 ~]$ cat /rest/sanblazes/1/fan1rpm
5221
[vlun@vlun-111 ~]$ cat /rest/sanblazes/1/fan1fail
0

Use * to batch the command. For example, to see the temperatures of all of the fans:

cat /rest/sanblazes/1/fan*temp

Example output (six fans, one temperature per line):

26
26
28
28
30
30

Status of the FAN Service

[vlun@vlun-111 ~]$ systemctl status sb_fans.service
sb_fans.service - SANBlaze Express NVMe Fan Control
   Loaded: loaded (/lib/systemd/system/sb_fans.service)
   Active: active (running) since Wed, 10 Oct 2018 11:36:14 -0400; 1 day and 23h ago
 Main PID: 7106 (sb_fans)
   CGroup: name=systemd:/system/sb_fans.service
           └ 7106 /usr/bin/sb_fans

Note: The service must be running for the above commands to contain accurate data.

Configuring Fan Temperature and Speed

Fans are controlled using two variables – temperature and speed – via the configuration file /etc/sb_fans.conf. There are four temperature variables T0–T3 and four speed variables S0–S3.

Fan Temperature

Fans will speed up or slow down as the temperature crosses into each temperature variable. At T4 or higher, fans will run at full speed. Default settings (in Celsius):

Variable

Value

T0

30

T1

40

T2

50

T3

65

Fan Speed

Speed zones are triggered by the temperature zones. At temperatures of T4 or higher, fans will run at full speed regardless of the speed zone settings. Default values (% of MAX RPM):

Variable

Value

S0

30

S1

40

S2

60

S3

80

All values are % of MAX RPM (100% = full speed, 0% = stopped fan).

Fan Airflow

By controlling the RPM of the fans, you can match the air flow in various chassis where the NVMe device is deployed. This is a valuable tool to characterize the thermal behavior of the device prior to deployment, or to simulate a device in a user’s environment.

The table below shows the approximate LFM (linear feet per minute) air flow rating based on different fan speed configurations for the industrial and commercial chassis. Measurements were taken with 16 same-height drives installed.

Air Flow Chart

% Speed set in /etc/sb_fans.conf

INLET LFM 15mm Drive

INLET LFM 7mm Drive

100

1373

875

90

1287

820

80

1096

698

70

985

627

60

880

561

50

708

451

40

554

353

30

450

287


Testing the Temperature of the PLX Chip (RM5 Only)

To test the temperature of the chip on the PLX switch, use the following command:

[root@x10-vlf ~]# sb_sdb -T
Temp = 41C

Reporting the Retrain (Recovery) Count

The retrain (recovery) count is part of -i and -l, or it can be shown on its own by using the register address bc4.

[root@x10-vlf ~]# sb_sdb -d 2 bc4
0x60810bc4: 0403005d

Displaying Port Modes

To read and display the port modes registers, add the -m flag to the sb_sdb command:

sb_sdb -d a -m

NVMe MI Commands and System Environmental Commands Using the sb_i2c Program

The SANBlaze SBExpress enclosure for testing NVMe drives can be monitored and controlled via a built-in program called sb_i2c. The sb_i2c program lets you monitor system environmental conditions such as voltage measure and margin, resets, hot plug, and PCIe status.

The sb_i2c program can also be used to build custom NVMe Management Interface (MI) commands. To build sb_i2c commands to go out on the wire, you can use the mi program.


Testing Surprise Removal of NVMe Devices

When testing your drives, you can test “surprise removal” without physically removing the drive. With SANBlaze you can do this remotely, right from your keyboard, without ever walking into the server room.

The following command removes power from the drive (surprise removal):

sb_i2c -d 6 -f POWER_ -w 1

Simulate that the Drive is Physically Gone

This command removes the PRESENT signal (as if the drive is now physically not there) in the SANBlaze-RM and RM5:

sb_i2c -d 6 -f IOX_DIS_PRSNT0_L -w 0

The background color behind slot number 6 will change to white (not yellow), indicating there is no present signal — the drive is “physically” gone according to the software.

Bringing the Drive Back

Enable present to bring the drive back:

sb_i2c -d 6 -f IOX_DIS_PRSNT0_L -w 1

To hot plug “button” the drive (graceful removal/replacement), assuming there is an actual drive in slot 6:

sb_i2c -d 6 -b

The slot color will change to GREEN indicating Present and Link, and the Pwr LED will be green indicating the drive has power.

Glitching PERST

Glitch PERST using -w 3 which does a quick on/off:

sb_i2c -d 6 -f IOX_PORT0_PERST_L -w 3
INFO: System 1 SBExpress-RM5 SN=930A9050002 Rev=R03 i2c=/dev/i2c-4
      AtlasPort=/dev/ttyACM0 MI_i2c=/dev/i2c-5 VLUN_Port=0
Feature IOX_PORT0_PERST_L value=3, oneshot action
INFO: 06L[0x62] 0x06[01:00] IOX_PORT0_PERST_L Def=0x01 Cur=0x00
INFO: 06L[0x62] 0x06[01:00] IOX_PORT0_PERST_L Def=0x01 Cur=0x01

-w 0 will hold the device in reset; -w 1 will release reset.

Glitching with HP_PERST

You can use HP_PERST to glitch across all risers:

# Glitch PCIe Reset signal on -RM or -DT5 riser card (IOX_PORT0_PERST_L) at slot 0 for 1 Second (1000000uS)
sb_i2c -d 0 -f IOX_PORT0_PERST_L -w 0 -g 1000000
sb_i2c -d 0 -f IOX_PORT1_PERST_L -w 0 -g 1000000

# Glitch PCIe Reset signal (HP_PERST_) at slot 0 for 1 Second
sb_i2c -d 0 -f HP_PERST_ -w 0 -g 1000000

List of sb_i2c Commands – Context Sensitive Help

Since iRisers in each slot can vary, context-sensitive help lets you check available commands per slot. Use the following syntax, where -d is the slot number, -V is verbose mode, and -f ? displays all available commands and their descriptions:

sb_i2c -d 0 -f ?

Example output:

ERROR: Unable to open device /dev/i2c-14 called from 2093
INFO: System 01[00] SBExpress SN=67d1bb62 Rev=R11 i2c_main=i2c-10 i2c_mi=i2c-11
      i2c_priv=i2c-14 SDB=/dev/ttyACM0 VLUN=0
WARN: Feature = not found on device type 600959003. Check CLI syntax

Signal Name              Default  Description
SW_DUALPRT_L             0x1      Reads or Writes the current state of Dual Port Switch
LNK_CUR_WIDTH            0x1      Reads the current state of PCIe Bus Width
LNK_CUR_SPEED            0x1      Reads the current state of PCIe Bus Speed
PRIMARY_BUS              0x1      Reads the PCIe Primary Bus Number
SECONDARY_BUS            0x1      Reads the PCIe Secondary Bus Number
LNK_CAP_SPEED            0x1      Reads the PCIe Speed Capability
LNK_CAP_WIDTH            0x1      Reads the PCIe Width Capability
LNK_CAP_ASPM             0x1      Reads the PCIe ASPM Capability
LNK_CAP_ENABLE           0x1      Reads the PCIe ASPM Enable
LNK_STATUS_CTRL_ASPM     0x1      Reads the PCIe ASPM Control
LNK_DISABLE              0x1      Reads the PCIe Link Disable
LNK_RETRAIN              0x1      Reads the PCIe Link Retrain
LNK_CUR_ACTIVE           0x1      Reads the PCIe Link Currently Active
LNK_TGT_SPEED            0x1      Reads or Writes the PCIe Link Target Speed
LNK_TGT_WIDTH            0x1      Reads or Writes the PCIe Link Target Speed
HP_PERST_|HP_PERST#      0x1      Alias names for Port0 or Port1 PCIe Reset
POWER|HP_PWREN|POWER_|POWER# 0x1  Alias names for power enable
HP_CLKEN_|HP_CLKEN#      0x1      Alias names for Port0 or Port1 Clock Enable
PERST_12V_L              0x1      Asserts PERST around Power Off/On

Signals for 600959 Gen5 EDSFF Riser:

Signal Name              Default  Description
DRIVE_PRSNT_L            0x1      Low indicates drive is present in the slot
DISABLE_3V3_AUX_L        0x1      Setting Low disables 3.3V power to DUT
DRIVE_EDSFF_LED_L        0x1      Setting Low will light EDSFF LED on DUT
TP16                     0x1      For internal use, connected to TP16
PWRDIS_L                 0x1      PWRDIS on EDSFF Connector P3 - Signal to drive, does not shut off riser power
SMRST_L                  0x1      Setting Low holds SMBus in Reset
EN_REFCLK_P0             0x1      Enable Reference Clock for Port0
EN_REFCLK_P1             0x1      Enable Reference Clock for Port1
DISABLE_12V_L            0x1      Setting Low shuts off 12V power to DUT
PERST_12V_L              0x1      Setting Low disables 12V power to DUT, power on asserts PERST
STATUS_BLUE_LED_L        0x1      Setting Low will light blue locate LED on Riser
TP17                     0x1      For internal use, connected to TP17
SW_DUALPRT_L             0x1      Setting Low enables dual port mode for selected slot
PORT0_PERST_L            0x1      Setting Low Asserts PERST to DUT Port 0
PORT1_PERST_L            0x1      Setting Low Asserts PORT1_PERST_L to DUT when in dual port mode
CLKREQ_L                 0x1      Setting Low Asserts CLKREQ_L to DUT when in single port mode

Usage Examples using slot number

Read voltage drive 2

sb_i2c -d 2 -m

Set the power supply to 11V

sb_i2c -v 11000

Read voltage, current and power on all drives

sb_i2c -d a -m

Hot-plug (button press) drive zero

sb_i2c -d 0 -b

Addressing the PLX by address, station, port and feature

I2C addresses are 0x38 and 0x39 for the -DT and 0x5d and 0x5e for the rackmount.

# Read link current speed on station 0x01, port 0x1b
sb_i2c -2 0x38 -s 0x01 -p 0x1b -f LNK_CUR_SPEED

# Upstream port (station 0x00, port 0x00)
sb_i2c -2 0x38 -s 0x00 -p 0x00 -f LNK_CUR_WIDTH

# Read register directly: Station 0x01, Port 0x10, Register 0xA8
sb_i2c -2 0x38 -s 0x01 -p 0x10 -a 0xa8

# Write 0x874813e9 to register 0xA8 (all 4 bytes must be specified)
sb_i2c -2 0x38 -s 0x01 -p 10 -a 0xa8 -a 0xa8 -w 87 48 13 e9

Notes:

  • -d -1 can be used to address PLX for Host 1 (-2 0x38)

  • -d -2 can be used to address PLX for Host 2 (-2 0x39)

  • When writing to registers directly, all 4 bytes must be specified.


Displaying Risers on the System

To show what type of riser is in a particular slot:

sb_i2c -d <slot #> -e

Example:

sb_i2c -d 1 -e

INFO: System 1 SBExpress-RM5 SN=930A9050005 Rev=R03 i2c=/dev/i2c-5
      AtlasPort=/dev/ttyACM0 MI_i2c=/dev/i2c-6 VLUN_Port=0
INFO: risertype=4000 i2caddr=50
INFO: Slot=1, Drivebay=1L device=600936000
INFO: Type=SBExpress-RM5 U.2 Gen4 Riser
INFO: Vendor=SANBlaze Technology, Inc.
INFO: Device=600936000
INFO: SN=936A9010030
INFO: Rev=R00
INFO: Insertions=0

Note: Slots with no drives installed may not be identified correctly.


Setting the Clock Frequency on a Slot

Use the script set_smbus_freq.sh, which takes a slot number and a clock frequency, finds the MI/SMBus I2C bus for that slot, and sets the clock frequency for that I2C bus. If no clock frequency is given, the current value will be shown and no changes will be made. If the clock frequency is given as 0, the default (100) will be used instead.


Signal to Tell a Drive to Power Down (PWRDIS)

PWRDIS is a signal on the U.2 and EDSFF connectors that can tell the drive to power down. SANBlaze supports this optional signal; however, not all drives support it. On RM5 the name is PWRDIS and on the DT5 the signal is IOX_PWRDIS.

Examples

EDSFF Riser (600959):

sb_i2c2 -d 0 -f help

INFO: System 01[00] SBExpress-RM5 SN=950A2050002 Rev=R01 i2c_main=i2c-8
      i2c_mi=i2c-7 i2c_priv=i2c-2 SDB=/dev/ttyACM4 VLUN=0

Signal Name              Default  Description
SW_DUALPRT_L             0x1      Reads the current state of Dual Port Switch

Signals for 600959 Gen5 EDSFF Riser:
DRIVE_PRSNT_L            0x1      Low indicates drive is present in the slot
DISABLE_3V3_AUX_L        0x1      Setting Low disables 3.3V power to DUT
DRIVE_EDSFF_LED_L        0x1      Setting Low will light EDSFF LED on DUT
TP16                     0x1      For internal use, connected to TP16
PWRDIS_L                 0x1      PWRDIS on EDSFF Connector P3 - Signal to drive, does not shut off riser power
SMRST_L                  0x1      Setting Low holds SMBus in Reset
EN_REFCLK_P0             0x1      Enable Reference Clock for Port0
EN_REFCLK_P1             0x1      Enable Reference Clock for Port1
DISABLE_12V_L            0x1      Setting Low shuts off 12V power to DUT
STATUS_BLUE_LED_L        0x1      Setting Low will light blue locate LED on Riser
TP17                     0x1      For internal use, connected to TP17
SW_DUALPRT_L             0x1      Setting Low enables dual port mode for selected slot
PORT0_PERST_L            0x1      Setting Low Asserts PERST to DUT Port 0
PORT1_PERST_L            0x1      Setting Low Asserts PORT1_PERST_L to DUT when in dual port mode
CLKREQ_L                 0x1      Setting Low Asserts CLKREQ_L to DUT when in single port mode

U.2 Riser (600957):

sb_i2c2 -d 0 -f help

INFO: System 01[00] SBExpress-RM5 SN=950A2080008 Rev=R04 i2c_main=i2c-12
      i2c_mi=i2c-11 i2c_priv=i2c-2 SDB=/dev/ttyACM6 VLUN=0

Signals for 600957 Gen5 U.2 Riser:
DRIVE_PRSNT_L            0x1      Low indicates drive is present in the slot
DISABLE_3V3_AUX_L        0x1      Setting Low disables 3.3V power to DUT
TP28                     0x1      For internal use, connected to TP28
PWRDIS_L                 0x1      PWRDIS on U.2 Connector P3 - Signal to drive, does not shut off riser power
SW_DUALPRT_L             0x1      Setting Low enables dual port mode for selected slot
EN_REFCLK_P0             0x1      Enable Reference Clock for Port0
EN_REFCLK_P1             0x1      Enable Reference Clock for Port1
DISABLE_12V_L            0x1      Setting Low disables 12V power to DUT
STATUS_BLUE_LED_L        0x1      Setting Low will light blue locate LED on Riser
NVME_CONN_P1             0x1      Connected to U.2 Connector P1
NVME_CONN_P2             0x1      Connected to U.2 Connector P2
PORT0_PERST_L            0x1      Setting Low holds PERST to DUT
PORT1_PERST_L            0x1      Setting Low Asserts PORT1_PERST_L to DUT when in dual port mode
CLKREQ_L                 0x1      Setting Low Asserts CLKREQ_L to DUT when in single port mode

M.2 Adapter and Riser Card Signal Information

This section shows you how to get signal information on the m.2 adapter (sb_i2c2 -d0 -U) and the riser card (sb_i2c2 -d0). The command -f ? gives help on “feature” (the signal name) and the -U command selects “upper” (the m.2 adapter on top of the riser).

Riser slot 0 (EDSFF, 600959):

sb_i2c2 -d 0 -f ?
INFO: Locating device eeprom on Riser slot 0 adapter=0
INFO: System 01[00] SBExpress-RM5 SN=950A2070010 Rev=R03 i2c_main=i2c-14
      i2c_mi=i2c-13 i2c_priv=i2c-4 SDB=/dev/ttyACM4 VLUN=0

(Signal table same as 600959 Gen5 EDSFF Riser above)

M.2 adapter on slot 0 (EDSFF to M.2 Gen5 Adapter, 600915):

sb_i2c2 -d 0 -U -f ?
INFO: Locating device eeprom on Riser slot 0 adapter=1

Available signals for 600915 EDSFF to M.2 Gen5 Adapter:
IOX_M2_CLKREQ_L          0x1      Assert CLKREQ_L M.2 connector Pin 52. Overrides host CLKREQ_L setting
IOX_M2_PEWAKE_L          0x1      Assert PEWAKE_L M.2 connector Pin 54
IOX_M2_I2C_PERST_L       0x1      Assert PERST# M.2 connector Pin 50
IOX_M2_CLEAR_PLA_L       0x1      Toggle Low then High to Clear PLA Latch
IOX_M2_3V3_PWR           0x1      Assert to enable VCC_3V3_M2 power to DUT
IOX_M2_ENABLE_I2C        0x1      Assert to enable SMBUS to DUT. Deassert to debug DUT I2C bus lockup
IOX_M2_PLA_DETECT_L      0x1      Latched Version of PLA from M.2 DUT. Use IOX_M2_CLEAR_PLA_L to clear
IOX_M2_PLN_L             0x1      Assert PLN_L M.2 Connector Pin 8
IOX_M2_READ              0x1      Read Bit[7:0] of the input register for the IO Expander (IOX) on the 600923000 adapter

Riser slot 1 (U.2 Gen5, 600957):

sb_i2c2 -d 1 -f ?
INFO: Locating device eeprom on Riser slot 1 adapter=0

(Signal table same as 600957 Gen5 U.2 Riser above)

M.2 adapter on slot 1 (U.2 to M.2 Gen5 Adapter, 600923):

sb_i2c2 -d 1 -U -f ?
INFO: Locating device eeprom on Riser slot 1 adapter=1

Available signals for 600923 U.2 to M.2 Gen5 Adapter:
IOX_M2_CLKREQ_L          0x1      Assert CLKREQ_L M.2 connector Pin 52. Overrides host CLKREQ_L setting
IOX_M2_PEWAKE_L          0x1      Assert PEWAKE_L M.2 connector Pin 54
IOX_M2_I2C_PERST_L       0x1      Assert PERST# M.2 connector Pin 50
IOX_M2_TP40              0x1      Assert TP40. Manufacturing use only
IOX_M2_3V3_PWR           0x1      Assert to enable VCC_3V3_M2 power to DUT
IOX_M2_ENABLE_I2C        0x1      Assert to enable SMBUS to DUT. Deassert to debug DUT I2C bus lockup
IOX_M2_PLA_L             0x1      Assert PLA_S3_L M.2 connector Pin 30
IOX_M2_PLN_L             0x1      Assert PLN_L M.2 Connector Pin 8
IOX_M2_READ              0x1      Read Bit[7:0] of the input register for the IO Expander (IOX) on the 600923000 adapter

SBExpress-RM5 Power On/Off Slots

For the SANBlaze SBExpress-RM5, sb_i2c2 is the preferred method. However, calling sb_i2c also works for compatibility.

Power off slot 5 on system 1:

sb_i2c  -n 1 -d 5 -f HP_PWREN -w 0
sb_i2c2 -n 1 -d 5 -f HP_PWREN -w 0

Power on slot 5 on system 1:

sb_i2c  -n 1 -d 5 -f HP_PWREN -w 1
sb_i2c2 -n 1 -d 5 -f HP_PWREN -w 1

List of sb_i2c2 Commands

[root@QA-RM5V2-100-178-IPMI-170 vlun]# sb_i2c2 -help
sb_i2c2: invalid option -- 'h'
sb_i2c2 program is under development. Currently supported features:

  -b               Press the button, to toggle the power to a slot
  -d <[slot range] | [-3]>
                   e.g. 1,2,3,6-9,34  Note: 34 selects MB, -d a selects 0-15,
                   -d A selects 16-31, -d aA selects 0-31
  -e               Read EEPROM. -d -3 for MB or -d N for slot
  -f <feature> [-w <value>]
                   Use -f ? for list of supported features on a given riser
  -g delay         Delay in uS between writing features for glitching,
                   e.g. sb_i2c2 -d 0 -f power -w 0 -g 1000000 -f power -w 1
  -i               Read current information from the slot
  -m [iterations] [delay]
                   Default iterations=1. 0=measure forever, minimum delay
                   between readings 10uS.
  -q               Quiet
  -v <voltage>     In mvolts, use -v 0 to calibrate
  -Z               Zero EEPROM to default values
  -I               Set the Vendor ID in specified EEPROM
  -D               Set the Device ID in specified EEPROM
  -S               Set the Serial Number in specified EEPROM
  -R               Set the Revision in specified EEPROM
  -C               Set the Chassis number in specified EEPROM
  -H               Set the cHassis serial number in specified EEPROM
  -E               Set the chassis rEvision in specified EEPROM
  -Q               Set up to 4 shunt values for this riser/adapter.
                   Provide -Q i,VALUE where:
                     i     = Shunt number 1-4
                     Value = Shunt resistance in uOhms, e.g. -Q 1,15000
  -T               Train the given slot to Gen1-5
  -W               Set lane width. ex: -W 2. Note some drives require PERST
                   to train back to x4. Drives must support up-configure.
  -x               i2c debug level 1
  -xx              i2c debug level 2
  -xxx             i2c debug level 3
                   NOTE: Internal use; echo 1 > /tmp/NVMe/gen6debug to enable Gen6 debug prints
  -U               Upper. Selects the M.2 adapter on a given slot
  -B               Background. Continue to run, don't exit. Will measure power on all slots
  -O               fOrce scan of adapter, ignore cache. Will force a re-read of riser and MB eeprom.
  -M [mode]        Retimer mode: mode=1 single port, mode=2 dual port

Help on what is supported by sb_i2c2:

sb_i2c2 -n 1 -d 5 -f ?

INFO: System 01[05] SBExpress-RM5 SN=950A110002 Rev=R01 i2c_main=i2c-12
      i2c_priv=i2c-8 SDB=/dev/ttyACM4 VLUN=0
INFO: Supported features:
INFO:   SLOT_VALID_IN_L
INFO:   SLOT_ID_IN_L
INFO:   DRIVE_PRSNT_L
INFO:   DISABLE_3V3_AUX
INFO:   PORT0_PERST_L
INFO:   PORT1_PERST_L
INFO:   DRIVE_EDSFF_LED
INFO:   RETIMER_RESET_3V3_L
INFO:   PWRDIS_L
INFO:   SMRST_L
INFO:   EN_REFCLK_P0
INFO:   EN_REFCLK_P1
INFO:   DISABLE_12V_L | HP_PWREN | POWER
INFO:   SLOT_ID0_L
INFO:   SLOT_ID1_L
INFO:   SLOT_ID2_L
INFO:   SLOT_ID3_L
INFO:   STATUS_BLUE_LED_L
INFO:   RETIMER_PERST_L
INFO:   SW_DUALPRT_L
INFO:   ID_VALID_L
INFO:   IOX_MAIN_G5X60_READ
INFO:   PRIVATE_G5X60_READ
INFO:   PRIVATE_G5X61_READ

List of PEX Commands

Use the -f ? command to get a list of the PEX features and commands:

sb_i2c -f ?

PEX Features are set and read by writing to registers in the PEX chips by station and port. The file /etc/pex8748.cfg describes accessible bit fields for registers. Users may define additional registers and fields; if the file is modified, it should be backed up.

Configuration file format (/etc/pex8748.cfg):

Field

Description

REGISTER

Hex value for register containing the feature (e.g., f70)

MSB

Most significant bit number (decimal int 0-31)

LSB

Least significant bit number (decimal int 0-31)

VALUE

Hex value to set in the register for the feature

ACTION

Name for action. Must be unique (e.g., PWR_LED_ON), used with -f

  • Read: [-f NAME] will read the current value of the register/feature

  • Default: [-f NAME -w] will write the default value from /etc/pex8748.cfg

  • Write: [-f NAME -w b1 b2 b3 b4] will write b1-4 to the named register/feature

Features for PLX switch (/etc/pex8748.cfg):

Version=005
# 001: Original
# 002: Add alternate feature name NAME_ for all NAME# features for REST compatibility
# 003: Add definition of register 0xF70
# 004: Add definition of register 0x80
# 005: Add definition of register 0x0C

REGISTER  MSB  LSB  DEFAULT  ACTION
00        31   16   0        DEVICE_ID
00        15   0    0        VENDOR_ID
0C        7    0    0        CACHE_LINE
10        31   0    0        BASE_ADDR_0
14        31   0    0        BASE_ADDR_1
18        7    0    0        PRIMARY_BUS
18        15   8    0        SECONDARY_BUS
18        23   16   0        SUBORDINATE_BUS
3C        22   22   0        SEC_BUS_RESET
80        0    0    0        ATTN_BUTTON
80        1    1    0        PWR_FLT_DETECTOR
80        2    2    0        MRL_SENSOR
80        3    3    0        PSNT_DET_CNG_ENA
80        4    4    0        CMD_CMPL_INT_ENA
80        5    5    0        HP_INT_ENA
80        7    6    3        ATTN_LED
80        9    8    3        POWER_LED
80        10   10   0        POWER# / POWER_
80        11   11   0        MECH_LATCH
80        12   12   0        STATE_CNG_ENA
80        16   16   0        HP_BUTTON
80        17   17   0        POWER_FLT
80        18   18   0        MECH_LATCH_STATE
80        19   19   0        PRESENCE_CNG
80        20   20   0        CMD_CMPL
80        21   21   0        MRL_SENSOR_STATE
80        22   22   0        PRESENCE
80        23   23   0        MECH_LOCK_STATUS
80        24   24   0        DATA_LINK_CNG
80        31   0    0        0x80
A8        31   16   8748     SUBSYS_ID
A8        15   0    13e9     SUBVEND_ID
208       31   0    0        PORT_DISABLE
260       31   0    0        EEROM_CTRL
264       31   0    0        EEROM_BUFFER
268       31   0    0        EEROM_CLOCK
270       31   0    0        EEROM_CRC
3AC       0    0    0        CONFIG_RELEASE
BAC       3    0    0        TEMP_TRIM
BAC       6    4    0        TEMP_RSV1
BAC       7    7    0        TEMP_ENABLE
BAC       15   8    0        TEMP_RSV2
BAC       26   16   0        TEMP_DATA
BAC       30   26   0        TEMP_RSV3
BAC       31   31   0        TEMP_VALID
BC4       31   0    0        RECOV_DIAG
BF0       31   0    0        RCV_ERRORS
F70       12   12   0        HP_SWCTRL
F70       13   13   0        HP_SWPWRGD
F70       20   20   1        HP_IO_RELOAD
F70       21   21   0        HP_PWRLED# / HP_PWRLED_
F70       22   22   0        HP_ATNLED# / HP_ATNLED_
F70       23   23   1        HP_PWREN
F70       24   24   0        HP_CLKEN# / HP_CLKEN_
F70       25   25   1        HP_PERST# / HP_PERST_
F70       26   26   0        HP_INTERLOCK
F70       27   27   0        HP_SW_PRSNT
F70       28   28   0        HP_SW_MRL
F70       29   29   0        PHY_PRT_CTRL_ENA
F70       31   0    0        0xf70 / 0xF70
FAC       31   0    0        BAD_TLP_CNT
FB0       31   0    0        BAD_DLLP_CNT
FB8       31   0    0        AER_UNCORR
FC4       31   0    0        AER_CORRECT
70        29   29   0        LNK_CTRL_ENA
74        3    0    3        LNK_CAP_SPEED
74        9    4    4        LNK_CAP_WIDTH
74        11   10   2        LNK_CAP_ASPM
74        20   20   1        LNK_CAP_ENABLE
78        1    0    0        LNK_STATUS_CTRL_ASPM
78        4    4    0        LNK_DISABLE
78        5    5    0        LNK_RETRAIN
78        19   16   3        LNK_CUR_SPEED
78        25   20   4        LNK_CUR_WIDTH
78        29   29   0        LNK_CUR_ACTIVE
98        3    0    3        LNK_TGT_SPEED
724       15   0    ffff     GEN3_FRM_ERR

IOExpander Features:

The system uses a PCA9551 LED Expander to control functionality on riser cards. The file /etc/ioexpander.cfg describes accessible bit fields for registers.

Configuration file format (/etc/ioexpander.cfg):

Field

Description

REGISTER

Hex value for register (e.g., 0x05, 0x06)

MSB

Most significant bit number (decimal int 0-7)

LSB

Least significant bit number (decimal int 0-7)

VALUE

Default hex value

ACTION

Must start with IOX_ and be unique

  • Read: [-f NAME] reads current value

  • Default: [-f NAME -w] writes default value

  • Write: [-f NAME -w b] writes byte b

# Hot Swap Buttons IOX_PORT1_HS can be set:
#   0=Pressed, 1=Unpressed, 2=Momentary
# Always use 2 for button press.
# _PERST_ signals can be set:
#   0=Held in Reset, 1=Reset released, 2=Momentary (500mS reset), 3=Oneshot (as fast as possible)

Example: sb_i2c -d 12 -f IOX_STATUS_LED -w 0        (0=ON, 1=OFF, 2=Rate1, 3=Rate2)
         sb_i2c -d 12 -f IOX_PWM0 -w -f IOX_PSC0 -w -f IOX_STATUS_LED -w 2  (blink at 1Hz)
         sb_i2c -d 12 -f IOX_PWM1 -w -f IOX_PSC1 -w -f IOX_STATUS_LED -w 3  (blink at 2Hz)

Features for IOExpander (/etc/ioexpander.cfg):

Version=007
# 001: Original
# 002: Add additional signal names for dual port riser
# 003: Add additional signal names for IOX devices on fan bus
# 004: Add additional signal names for IOX devices on U.2 to M.2 adapter
# 006: Add additional signal names for IOX devices
# 007: IOX_CLKREQ_L is now IOX_PWRDIS_L due to spec change

REGISTER  MSB  LSB  DEFAULT  ACTION
01        7    0    25       IOX_PSC0
02        7    0    80       IOX_PWM0
03        7    0    12       IOX_PSC1
04        7    0    80       IOX_PWM1
05        1    0    1        IOX_PWRDIS_L
05        3    2    1        IOX_NVME_PIN2
05        5    4    1        IOX_NVME_PIN1
05        7    6    1        IOX_DIS_PRSNT0_L
06        1    0    1        IOX_STATUS_LED
06        3    2    1        IOX_PORT0_HS
06        5    4    1        IOX_PORT1_HS
06        7    6    1        IOX_PORT0_PRSNT_L
05        1    0    1        IOX_PORT1_PRSNT_L
05        3    2    1        IOX_DIS_PRSNT1_L
05        5    4    1        IOX_7V_POWER
05        7    6    1        IOX_NC1
06        1    0    1        IOX_PORT0_PERST_L
06        3    2    1        IOX_PORT0_100MS_RST
06        5    4    1        IOX_PORT1_PERST_L
06        7    6    1        IOX_PORT1_100MS_RST
05        1    0    1        IOX_I2C_MAIN_RST_L
00        1    1    1        IOX_NC2
00        2    2    1        IOX_GE1_DCPWR_OK_L
00        3    3    1        IOX_GE2_DCPWR_OK_L
00        4    4    1        IOX_NC3
00        5    5    1        IOX_PEX_STRAP_VS_MODE0_3V3
00        6    6    1        IOX_SLOTS_PWR_ON
06        7    6    1        IOX_SLOTS_PFON_L
00        0    0    1        IOX_SW_PEX_EE_SEL_0
00        1    1    1        IOX_SW_PEX_EE_SEL_1
00        2    2    1        IOX_SW_PEX_EE_SEL_2
00        3    3    1        IOX_SW_SLOTS_PWR_ON
00        4    4    1        IOX_SW_FORCE_POWER_ON_L
00        5    5    1        IOX_SW_BLK_CBL_RST
00        6    6    1        IOX_SW_DIS_USB_L
00        7    7    1        IOX_SW_ONBOARD_CLKSEL
05        1    0    1        IOX_M2_PWRDIS_L
05        3    2    1        IOX_M2_PEWAKE_L
05        5    4    1        IOX_M2_I2C_PERST_L
05        7    6    1        IOX_M2_NC1
06        1    0    1        IOX_M2_STATUS_L
06        3    2    1        IOX_M2_ENABLE_I2C
06        5    4    1        IOX_M2_NC2
06        7    6    1        IOX_M2_NC3_H
05        1    0    1        IOX_MB_I2C_FAN_RST_L
05        3    2    1        IOX_MB_I2C_XSLT_RST_L
05        5    4    1        IOX_MB_SLOT_I2C_RST_L
05        7    6    1        IOX_MB_NC3
06        1    0    1        IOX_MB_NC4
06        3    2    1        IOX_MB_NC5
06        5    4    1        IOX_MB_NC6
06        7    6    1        IOX_MB_NC7

Displaying Reset Time on /iportX/target File

The following timing values are located in /iportX/targetY:

TimeLinkUpToConfigReady=Nus
TimeProbeToEnable=Nus
TimeLinkUpToEnable=Nus
TimeEnableToReady=Nus
TimeDisableToNotReady=Nus
TimeReadyToIdentify=Nus
TimeShutdown=Nus
TimePCIeLinkDownUp=Nus
TimePCIeFLR=Nus
TimePowerOnToLinkUp=Nus
TimeLinkUpToProbe=Nus
TimeLinkUpToMediaAccess=Nus
TimeProbeToMediaAccess=Nus
TimeReadyToMediaAccess=Nus

All values N are decimal; a line is only printed if N is not 0.

Example:

$ grep ^Time[A-Z] /iport0/target103
TimeProbeToEnable=102257us
TimeLinkUpToEnable=210282us
TimeEnableToReady=93us
TimeReadyToIdentify=148us
TimePCIeLinkDownUp=31553us
TimePCIeFLR=5976us
TimeLinkUpToMediaAccess=222489us
TimeReadyToMediaAccess=16360us

To clear all cached timings (write-only operation):

echo ClearTimings > /iportX/targetY

Example:

[root@100-130-IPMI-100-116 ~]# grep ^Time[A-Z] /iport0/target103
TimeProbeToEnable=104557us
TimeEnableToReady=98us
TimeReadyToIdentify=489us

[root@100-130-IPMI-100-116 ~]# echo ClearTimings > /iport0/target103

[root@100-130-IPMI-100-116 ~]# grep ^Time[A-Z] /iport0/target103
[root@100-130-IPMI-100-116 ~]#

Management Interface (MI) Program

The sb_i2c program can also be used to build custom NVMe MI commands. To build sb_i2c commands to go out on the wire, use the mi program. Use -? to see the mi options:

mi -?

usage: mi [switches] <opcode> [<subopcode>]

switches:
  -i   VLUN path (/iportX/targetYlunZ) for inband transport
  -a   PCIe path (0000:XX:YY.Z) for vdm transport
  -s   system (1 to 8)
  -d   drive (0 to 31)
  -t   type (-1=Basic, 0=Control Primitive, 1=NVMe-MI Command,
             2=NVMe Admin Command, 4=PCIe Command)
  -p   port identifier
  -Q   low-level SMBus access; translates to -1 for sb_i2c and sb_i2c2
  -c   controller identifier
  -n   namespace identifier
  -o   offset
  -l   length
  -m   MI request (pairs of hex digits for each byte)
  -w   MI data (pairs of hex digits for each byte)
  -W   name (of file of write data)
  -k   timeout in seconds
  -r   raw
  -S   Command Slot Identifier (CSI) value for the MI header
  -T   transport (smbus, vdm, or inband)
  -f   force
  -v   verbose
  -L   loopback

Note: In Basic mode, the “opcode” is the offset to start reading at (or use -o <offset>). For offset 0, the length defaults to 8. For offset 8, the length defaults to 24. For all other offsets, the length defaults to 32. The -p, -c, -n, -o, -l, opcode, and subopcode arguments can be given in either decimal (default) or hex (by prepending 0x).

Example — Get NVMe subsystem information for drive in slot 5:

mi -t 1 -s 1 -d 5 0 0

Read NVMe-MI Data Structure
command is 'sb_i2c -n 1 -d 5 -z -w 84 08 00 00 00 00 00 00 00 00 00 00 00 00 00 00 e2 00 06 07'
Response Data Length: 32
NVM Subsystem Information Data Structure:
  Number of Ports: 3
  NVMe-MI Major Version Number: 1
  NVMe-MI Minor Version Number: 0

You can copy and paste the displayed sb_i2c command (minus the last four CRC bytes) and execute it in the CLI to see the returned data. Add -V for verbose mode to see the complete SMBus packet hex bytes sent on the wire:

smbus_req[0]:
3a 0f 19 45 01 00 00 c8 84 08 00 00 00 00 00 00
00 00 00 00 00 00 00 00 e2 00 06 07 de

The first 8 bytes are the MCTP header, which can be included in the sb_i2c command directly.

NVMe Subsystem Reset

Use opcode 7, with the subopcode as reset type (0 = NSSR):

mi -t 1 -s 1 -d 8 7 0

Reset
command is 'sb_i2c -t -n 1 -d 8 -z -w 84 08 00 00 07 00 00 00 00 00 00 00 00 00 00 00 72 2c 53 91'
no response received

In-band MI Support for NVMe Initiator and NVMf Target

Use the inband path with -i (/iportX/targetYlunZ). If -i is specified, it overrides -s and -d, and instead of using sb_i2c, it uses the I/O program.

Examples:

# Read NVMe-MI Data Structure (subsystem info)
mi -i /iport4/target0lun1 -t 1 0 0

Read NVMe-MI Data Structure
command is 'io /iport4/target0lun1 SendReceiveMI -mi 84080000000000000000000000000000e2000607'
Response Data Length: 32
NVM Subsystem Information Data Structure:
  Number of Ports: 1
  NVMe-MI Major Version Number: 1
  NVMe-MI Minor Version Number: 1

# Read Port Information
mi -i /iport4/target0lun1 -t 1 0 1

Response Data Length: 32
Port Information Data Structure:
  Port Type: 1
  MCTP MTU: 0
  PCIe Max Payload Size: 1 (256)
  PCIe Supported Link Speeds: 3h
  PCIe Current Link Speed: 2h
  PCIe Maximum Link Width: 8h
  PCIe Negotiated Link Width: 8h
  PCIe Port Number: 0ah

# NVM Subsystem Health Status Poll
mi -i /iport4/target0lun1 -t 1 1

NVM Subsystem Health Data Structure:
  NVM Subsystem Status:
    Drive Functional: 1
    Reset Not Required: 1
    Port 0 PCIe Link Active: 1
    Port 1 PCIe Link Active: 0
    SMART Warnings: 0
  Composite Temperature: 38 degrees C
  Percentage Drive Life Used: 5%
  Composite Controller Health Status:
    Critical Warning: 0
    Available Spare: 0
    Percentage Used: 0
    Composite Temperature Change: 0
    Controller Status Change: 0
    Firmware Activated: 0
    Namespace Attribute Changed: 0
    Controller Enable Change Occurred: 0
    NVM Subsystem Reset Occurred: 0
    Shutdown Status: 0
    Controller Fatal Status: 0
    Ready: 1

©Copyright 2025 SANBlaze Technologies, Inc.