2 Port 10GbE RoCE SFP+ PCIe 3.0 Adapter
(FC:
EC37, EC38 & EL3X)
2 Port 10GbE RoCE SR PCIe
3.0 Adapter
(FC:
EC2M, EC2N & EL40)
******* PLEASE
READ THIS ENTIRE NOTICE *********
DATE: February 6, 2017
1.0 Microcode and
Document Revision History
7.0
Determine the Current Microcode Level for AIX
8.0 Downloading the RPM Format File to the
Target Server for AIX
9.0 Discovery Tool
Microcode CD-ROM creation and download instructions
10.0 Verifying
microcode before download and notes for AIX
11.0 Microcode Download Procedure for AIX
11.1 Setting up for Microcode download
11.2 Downloading Microcode to the
Adapter
11.3 Verify and Re-configure adapters
Firmware Level |
Description |
2.40.5030 /
000200405030 |
Impact: Usability Severity: ATT - Fixed an issue where outbound
completions were blocked during driver restart |
2.36.5000 /
000200365000 |
- Fixed length calculated of UDP packets,
which resulted incorrect UPD length causing miscalculation of CRC. - Fixed completion error issue
when ECN was enabled. - Fixed a race condition that
caused duplicated read request. - Fixed an issue causing packets
to get stuck when driver send pause frame with dmac
equal to device's mac address. |
2.34.5000 /
000200345000 |
New Release |
2.33.5000 / 000200335000 |
Original Release |
Document Revision History |
Description |
V4.0 – 2/3/2017 |
Version 2.40.5030
release |
V3.0 – 9/19/2016 |
Version 2.36.5000
release |
V2.0 – 8/5/2015 |
New Release |
V1.0 – 1/26/2015 |
Original Release |
|
|
This Readme file is intended to give directions on how to update the
microcode found on the 2 Port
10GbE RoCE SFP+ PCIe 3.0 Adapter and 2 Port 10GbE RoCE SR PCIe 3.0 Adapter
Non-Concurrent Download
The microcode installation does NOT support concurrent download. The entX devices can be used during and after the download, but
update will not go into effect until either the device is reloaded using rmdev/cfgmgr or a reboot is
performed.
NOTE: It is recommended that the installation be scheduled during a
maintenance window or during non-peak production periods.
Approximately 20 minutes.
Feature Code EC37 and EC2M:
8284-22A, 9119-MHE, 9119-MME
Feature Code EL3X and EL40:
8247-21L, 8247-22L
Feature Code EC37 and EC2M:
8247-21L, 8247-22L, 8284-22A, 8286-41A, 8286-42A,
8408-E8E, 9119-MHE, 9119-MME
For Linux operating systems, use the following
procedure:
1. Find the PCI slot the ConnectX-3 adapter is
plugged into. Issue the following command:
lspci | grep "ConnectX-3 Pro"
For example:
# lspci | grep
"ConnectX-3 Pro"
0000:50:00.0 Ethernet controller: Mellanox Technologies MT27520 Family
[ConnectX-3 Pro]
This tells us that the adapter is in the PCIe slot 0000:50:00.0 which will be needed in the
following steps.
2. Start MFT tools by running “mst
start”
#mst
start
Starting MST (Mellanox Software Tools) driver set
Loading MST PCI module - Success
Loading MST PCI configuration module - Success
Create devices
If the system does not have the mst command available, please install the Mellanox
Firmware Tools (MFT) available here: http://www.mellanox.com/page/management_tools
3. Verify the current Firmware level by running “flint -d <pci bus ID> -q”. Replace <pci bus ID> with the ID found in the lscpi command from step 2.
# flint -d 0000:50:00.0 q
Image type: FS2
FW Version: 2.36.5000
FW Release Date: 23.12.2015
Product Version: 02.33.50.00
Rom Info: type=FCODE
version=0.0.170 devid=4103 proto=ETH
Device ID: 4103
Description: Node Port1 Port2 Sys image
GUIDs: ffffffffffffffff ffffffffffffffff
ffffffffffffffff ffffffffffffffff
MACs:
f45214751ed0 f45214751ed1
VSD:
PSID: IBM1200111023
This command reveals the current
version of the FW; which is 2.36.5000 in this particular case. If the version is less than 2.40.5030, please update.
Please note the value for PSID. This will determine which image is needed in
later steps.
4. Download the firmware RPM file to the
destination machine.
Use this method to download the new
microcode to a Linux system:
NOTE: The instructions that follow are specific Linux
commands. Linux commands are CASE (lower and upper) SENSITIVE, and must
be entered exactly as shown, including filenames.
rpm -ivh /tmp/000200405030.Linux.rpm
The microcode package will install the
firmware images in the /lib/firmware directory. If a message is displayed
saying the "package <package_name> is
already installed", you will need to uninstall the listed rpm package. On
the command line type:
rpm -e <package_name>
where <package_name>
is the name of the package that was returned in the message. Return to Step 2
and attempt to install the file again.
5. Verify the contents of the image before flashing:
ls -l /lib/firmware/
b31507101410??04.000200405030to verify file size:
·
b31507101410e704.000200405030 = 899712
·
b31507101410eb04.000200405030= 899708
sum /lib/firmware/
b31507101410??04.000200365000 to verify Checksum:
·
b31507101410e704.000200405030= 16064
·
b31507101410eb04.000200405030= 62002
6. Update the FW with "flint -d <pci bus id> -i <image
filename> burn"
NOTE: The PSID value from step 3 will
determine which file to use for firmware update.
# flint -d
0000:50:00.0 -i b31507101410e704.000200405030
burn
Current FW version on flash: 2.36.5000
New FW version: 2.40.5030
Burning FW image without
signatures - OK
Restoring signature - OK
If you receive the following error message, it means the wrong file was
selected for update, and you need to choose the other filename.
-E- PSID mismatch.
The PSID on flash (IBM1200111023) differs from the PSID in the given image (IBM1200113023).
7. At this point we have burned the new FW on to
the Mellanox adapter. However, this FW will not take effect till the system is
rebooted.
Reloading the drivers is not enough; it requires
the system to be rebooted.
Basic card function is supported on:
·
AIX
Version 6.1 with the 6100-09 Technology Level and service pack 5 and APAR
IV68443 or later
·
AIX
Version 6.1 with the 6100-08 Technology Level and service Pack 7 or later
(planned availability 9/30/2015)
·
AIX Version
7.1 with the 7100-03 Technology Level and Service Pack 5 and APAR IV68444 or
later
·
AIX
Version 7.1 with the 7100-02 Technology Level and Service Pack 7 or later (planned availability 9/30/2015)
If you are using another release of AIX, ensure that the adapter is
supported on that release before you install the adapter. Contact service and
support for assistance.
Before you install the microcode, it is important to determine the
microcode level of the Adapter installed in the target system. Use the
following instructions to read the ROM level stored in the Adapter's VPD.
A.
List the RoCE adapters installed in the system by
typing:
lsdev | grep
RoCE
B.
To check the current microcode level for the
adapter or controller enter the following command:
lsmcode -cd entX
Where "X" is the instance of the adapter. The command will
produce output similar to:
# lsmcode -cd ent4
The current
microcode level for ent4 is
000200365000
If the ROM Level is less than 000200405030 you should update the microcode.
Use
this method to download to an AIX system:
NOTE: The instructions that follow are specific AIX commands.
AIX commands are CASE (lower and upper) SENSITIVE, and must be
entered exactly as shown, including filenames.
A.
Make two directories on your AIX system to receive
the RPM format file.
Enter: "mkdir /tmp/microcode"
and then create this directory
Enter: "mkdir /tmp/microcode/RPM"
B.
Transfer the RPM format file to the /tmp/microcode/RPM directory (using "Save as
..."). Change to that directory, "cd /tmp/microcode/RPM".
You'll see the filename for the RPM file.
"rpm -ihv --ignoreos 000200405030.aix.rpm"
C.
For AIX: The microcode files will be added to
/etc/microcode/.
D.
Two microcodes file will
be copied to "/etc/microcode". The file size and checksum of the microcode
image will be verified in Section 10.0.
File Names:
b31507101410e704.000200405030
b31507101410eb04.000200405030
NOTE:
- "/etc/microcode" is a symbolic link
to "/usr/lib/microcode".
- If permission does not allow the copy to the above stated directory or
file then the user will be prompted for a
new location.
To obtain information how to burn a CD-ROM and run the Discovery Tool
for an AIX or Linux System please go to:
http://www-304.ibm.com/webapp/set2/firmware/lgjsn?mode=10&page=cdrom.html
A.
After running the Discovery Tool successfully the
"/tmp/microcode/RPM" directory was created
and your rpm files are copied from the CD-ROM.
B.
Change to that directory, "cd /tmp/microcode/RPM".
C.
Unpack the file by executing the instructions
below:
Enter the command:
"rpm -ihv --ignoreos 000200365000.aix.rpm"
D.
Two microcodes file will
be copied to "/etc/microcode". The file size and checksum of the microcode
image will be verified in Section 10.0.
File Names:
b31507101410e704.000200405030
b31507101410eb04.000200405030
NOTE:
- "/etc/microcode" is a symbolic
link to "/usr/lib/microcode".
- If permission does not allow the copy to the above stated directory or
file then the user will be prompted for a new location.
- For customers using the AIX Diagnostics CD, please refer to the IBM
System Hardware information Center for instructions.
Please verify the file size and checksum of the raw microcode files
matches what is listed below.
ls -l /etc/microcode/b31507101410??04.000200405030 to verify file size:
·
b31507101410e704.000200405030= 899712
·
b31507101410eb04.000200405030= 899708
sum /etc/microcode/b31507101410??04.000200405030
to verify Checksum:
·
b31507101410e704.000200405030= 16064
·
b31507101410eb04.000200405030= 62002
A.
Stop all applications that use this
interface/adapter.
B.
Remove the interface/IP address from the all ports
identified in section 7.0 for the adapters that will be upgraded.
a.
Before detaching the interface, record the IP
address and any other pertinent information that was configured on the
Adapter. This information may be needed
if the microcode update overwrites this section on the Adapter.
C.
If the interfaces are members of an SEA, the SEA
devices must be moved to a defined state.
a.
"rmdev
-l enX" - where "X" is the interface
number for the Shared Ethernet Adapter.
b.
"rmdev
-l entX" - where "X" is the interface
number for the Shared Ethernet Adapter.
D.
If the interfaces are members of an EtherChannel, the EtherChannel
device must be moved to a define state.
a.
"rmdev
-l enX" - where "X” is the interface number
for the EtherChannel adapter.
b.
"rmdev
-l entX" - where "X" is the interface
number for the EtherChannel adapter.
E.
For every port associated with the adapter, the enX interfaces must be changed to a defined state.
a.
"rmdev
-l enX" - where "X" is the interface
number for the adapter port.
b.
This command will be run 2 times, once for each
port on the adapter.
A.
At the command line type "diag"
B.
Select the "Task Selection" from
diagnostics menu.
C.
Select "Microcode Tasks” then select “Download
Microcode" from the menu.
D.
Select all of the entX
RoCE Converged Network Adapters that need to be updated from the list of
devices by using the arrow keys to highlight the entry and pressing
"Enter" to mark it. Press "F7" or "ESC+7"
when you are done marking all the adapters you want to flash.
E.
If a source selection menu is displayed, Select
"/etc/microcode".
F.
A dialogue box may be displayed on screen. It
will state that the current microcode level on the adapter is not in the /etc/microcode directory. This is acceptable because
the adapter will reject any incorrect code. Press "Enter" to
continue.
G.
Select 000200405030 level and press
"Enter" to flash the adapter.
H.
The following message will appear on the screen
when download is completed: "Microcode download complete
successfully. The current microcode level for the device is ...
Please run diagnostics on the adapter to ensure that it is functioning
properly."
I.
If you selected more than one adapter to update,
then steps 6-9 will repeat until all adapters are updated.
J.
Exit diagnostics.
A.
Run "cfgmgr" to
reconfigure the adapters that were moved to defined
before the update.
B.
Verify the code level is 000200405030
by typing "lsmcode
-cd entX" for each adapter updated, where "X"
is the instance of the RoCE adapter.