Mellanox Technologies MIS5022Q-1BRR Manuel utilisateur

www.mellanox.com

InfiniScale® IV 8-Port QSFP 40Gb/s InfiniBand

Switch User Manual

P/N:MIS5022Q-1BRR, MIS5022Q-1BFR, MIS5024Q-1BRR,

MIS5024Q-1BFR, MIS5022Q-1BFR, MIS5022Q-1BRR,

MIS5022Q-1BFC, MIS5022Q-1BRC

Rev 1.2

Mellanox Technologies

350 Oakmead Parkway Suite 100

Sunnyvale, CA 94085

U.S.A.

www.mellanox.com

Tel: (408) 970-3400

Fax: (408) 970-3403

Mellanox Technologies, Ltd.

PO Box 586 Hermon Building

Yokneam 20692

Israel

Tel: +972-4-909-7200

Fax: +972-4-959-3245

Mellanox, BridgeX, ConnectX, Virtual Protocol Interconnect, InfiniBlast, InfiniBridge, InfiniHost, InfiniRISC, InfiniScale, and InfiniPCI

are registered trademarks of Mellanox Technologies, Ltd. CORE-Direct, FabricIT, and PhyX are trademarks of Mellanox Technologies, Ltd.

All other trademarks are property of their respective owners.

All other marks and names mentioned herein may be trademarks of their respective companies.

QDR InfiniBand Switch Platform User Manual

Document Number: 3373

Rev 1.2

Mellanox Technologies

NOTE:

THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED DOCUMENTATION ARE

PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE

OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE

CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX

TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX

TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE

HIGHEST QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED

WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE

DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT,

INDIRECT, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND (INCLUDING, BUT NOT LIMITED TO,

PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS

INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORYOF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,

OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE PRODUCT(S) AND

RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

QDR InfiniBand Switch Platform User Manual Rev 1.2

Mellanox Technologies

Contents

Contents 3

List of Figures 5

List of Tables 6

Revision History 7

About this Manual 8

Intended Audience 8

Related Documentation 8

Conventions 8

Mellanox Part Numbering Legend 9

Chapter 1 Overview 10

1.1 Serial Number and Product Version Information 10

Chapter 2 Installation Safety Warnings 11

Chapter 3 Externally Managed Switches 14

Chapter 4 Hardware Basic Operation and Installation 15

4.1 Switch Platform Hardware Overview 15

4.1.1 LED Assignments 15

4.2 AirFlow 18

4.3 QSFP Cable Power Budget Classification 18

4.4 Interfaces 19

4.4.1 Port Connector Interfaces 19

4.5 Management andFW Updating Interfaces 19

4.5.1 RJ-45 Connector (I2C) 19

Chapter 5 Installation 21

5.1 PackageContents 21

5.2 Installing the Switch in the Rack 22

5.2.1 Table Top Installation 22

5.2.2 Installation Procedure for a Single Switch Center-of-Rack Installation 23

5.2.3 Installation Procedure for a Side by Side Installation 24

5.2.4 Grounding the Switch 27

5.2.5 Power Connections and Initial Power On 27

5.2.6 InfiniBand Cable Installation 27

5.3 Disassembly of the Switch from the Rack 28

5.3.1 Disassembly Procedure — Single Switch Centered Installation 28

5.3.2 Disassembly Procedure — Side by Side Installation 29

5.4 Disposal 29

5.5 UpdatingFirmware 29

5.5.1 Current Firmware Revision 29

5.5.2 How to Get Mellanox Firmware Tools (MFT) 32

5.5.3 Open SM 32

Chapter 6 Troubleshooting 33

Appendix A Specification 35

A.1 EMC Certification Statements 36

A.2 VCCI Statements (Japan) 39

A.3 MIC Certification (Korea) 39

Appendix B QSFP Interface 40

Appendix C RJ45 I2C Interface 42

Rev 1.2

Mellanox Technologies

Appendix D Replacement Parts Ordering Numbers 43

Appendix E Avertissements de sécurité d’installation (French) 44

Appendix F Installation - Sicherheitshinweise (German) 46

Appendix G Advertencias de seguridad para la instalación (Spanish) 48

Appendix H Special Regulations Regarding Finland, Sweden, Denmark, and Norway 51

QDR InfiniBand Switch Platform User Manual Rev 1.2

Mellanox Technologies

List of Figures

Figure 1: Generic Product label 10

Figure 2: Switch System Front Panel 15

Figure 3: Status LEDs 16

Figure 4: Port Numbering 19

Figure 5: Top and Bottom Ports 19

Figure 6: RJ45 I2C Connector 20

Figure 7: Placing the Bumpers 22

Figure 8: Installation Kit for a Single Centered Installation 23

Figure 9: Connect the Mounting Brackets 24

Figure 10: Installation Kit Parts for a Side by Side Installation 25

Figure 11: Screw on the Switch Mounted Rails 25

Figure 12: Caged Nut Placement 26

Figure 13: Front Panel Orientation 26

Figure 14: Top and Bottom Ports 28

Figure 15: MTUSB-1 with Cables 31

Figure 16: I2C Cable Connected to IS5022 31

Figure 17: QSFP Male and Female Connections 41

Rev 1.2

Mellanox Technologies

List of Tables

Table 1: Revision History Table 7

Table 2: Reference Documents 8

Table 3: Switch Status LED Configurations 16

Table 4: Fan LED Configurations 17

Table 5: Bad Port LED Configurations 17

Table 6: Connector Physical and Logical Link Indications 18

Table 7: Installation Kit Options 21

Table 8: IS5022 Specification Data 35

Table 9: Switch Certification Status 36

Table 10: InfiniBand QSFP Connector Pinout 40

Table 11: Replacement Parts Ordering Numbers 43

QDR InfiniBand Switch Platform User Manual Rev 1.2

Mellanox Technologies

Revision History

Table 1 - Revision History Table

Date Revision Description

Mar. 2011 Rev 1.2 Added note and figure to blank panel installtion step.

Made changes to the installation process for the two switch frame.

Made changes to FW burning sections.

Feb. 2011 Rev 1.1 Removed UID LED support

Jan. 2011 Rev 1.0 Initial Release

Rev 1.2

Mellanox Technologies

About this Manual

This manual describes the installation and basic use of the Mellanox IS5022 switch, which is

based on the InfiniScale® IV InfiniBand switch device.

Intended Audience

This manual is intended for users and system administrators responsible for installing and setting

up the switch platforms listed above.

The manual assumes familiarity with the InfiniBand® Architecture Specification.

Related Documentation

Additional Documentation available from Mellanox:

Conventions

Throughout this manual, the name IS5022 and the term switch are used to describe the 8-port

QSFP 40Gb/s InfiniBand Switch unless explicitly indicated otherwise.

Table 2 - Reference Documents

Switch Firmware and

Firmware Update

Tools

See

http://www.mellanox.com > Support > Download Firmware Tools

Note that the Switch System described in this manual is based on Mellanox

Technologies’ InfiniScale

IV switch device.

Mellanox OFED

Stack for Linux

User’s Manual

See

http://www.mellanox.com > Support > InfiniBand Software and Drivers

Click “Mellanox OpenFabrics Enterprise Distribution for Linux

(MLNX_OFED)”

Select the Linux User’s Manual

The embedded OS and tools on the CPU in the management module is a

subset of the Mellanox OFED stack.

Mellanox Firmware

Tools (MFT)

User’s Manual

Document # 2329

The MFT (Mellanox Firmware Tools) package is a set of firmware tools.

The manual supplied with this package provides an overview of the firm-

ware its installation and replacement. The MFT can be downloaded with its

documentation at:

http://www.mellanox.com > Support > Download Firmware Tools

QDR InfiniBand Switch Platform User Manual Rev 1.2

Mellanox Technologies

Mellanox Part Numbering Legend

This symbol indicates information that is helpful to the user.

BEWARE! This symbol indicates a situation that can potentially cause personal injury

or damage to hardware or software.

Place Field Decoder

Mellanox Technologies

System Type InfiniScale Switch

Model Family

Form factor 22 = 8 Ports

Separator

Power supplies 1 = 1 power supply

Depth of the Unit B = short depth

Air Flow direc-

tion R= Connector side to rear side airflow

F= Rear side to connector side airflow

RoHS R=RoHS6

Overview

Rev 1.2

Mellanox Technologies

1Overview

This User Manual provides an overview of the InfiniBand switch product, an 8-port switch capa-

ble of up to 640Gb/s of non-blocking switching capacity. Based on the fourth generation InfiniS-

cale IV InfiniBand switch device. The platform delivers industry leading bandwidth, latency, and

scalability in a half width 1U form factor. The platform can be tiered to build larger multi stage

networks in a variety of topologies such as Fat-Tree, Mesh, 3D Torus etc. All port configurations

are covered in this manual which include switches for QSFP InfiniBand ports.

Mellanox IS5022 switch systems are an ideal choice for smaller departmental or back-end cluster-

ing uses with high-performance needs, such as storage, data base and GPGPU clusters. Networks

built with IS5022 systems can carry converged traffic with the combination of assured bandwidth

and granular quality of service. IS5022 systems provide up to 40Gb/s full bidirectional bandwidth

per port.

The switch platform comes pre-installed with all necessary firmware and is configured for stan-

dard operation within an InfiniBand fabric. This switch requires an InfiniBand compliant Subnet

Manager running from one of the hosts or another managed switch. All that is required for normal

operation is to follow the usual precautions for installation and to connect the switch to the HCAs

or other switches within the fabric. Once connected, the Subnet Management software automati-

cally configures and begins utilizing the switch.

It is recommended that the Mellanox OpenFabrics software package be installed on all nodes con-

nected to the IS5022. The software package provides a subnet manager and network management

tools as well as connectivity software for servers and storage, and is available on the Mellanox

web site. See Chapter 3 for more information.

Basic installation and hardware maintenance is covered in “Hardware Basic Operation and Instal-

lation” on page 15.

1.1 Serial Number and Product Version Information

The Serial number and GUID for the switch are found on the front panel below the Mellanox logo.

The serial number and product version information are found on the label seen in the figure below.

Figure 1: Generic Product label

Also on this label is the GUID identifier for the switch.

Ce manuel convient aux modèles suivants

Table des matières

Autres manuels Mellanox Technologies Changer

Mellanox Technologies

Mellanox Technologies SX6710 Instructions d'utilisation

Mellanox Technologies

Mellanox Technologies MSX6536-10R Manuel

Mellanox Technologies

Mellanox Technologies MIS5035Q-2SFC Manuel utilisateur

Mellanox Technologies

Mellanox Technologies SX6720 Instructions d'utilisation

Mellanox Technologies

Mellanox Technologies Spectrum SN2700 Instructions d'utilisation

Mellanox Technologies

Mellanox Technologies MSX6536-10R Manuel

Mellanox Technologies

Mellanox Technologies SX1710 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies MSX6512-NR Manuel

Mellanox Technologies

Mellanox Technologies TX6100 MetroX Series Instructions d'utilisation

Mellanox Technologies

Mellanox Technologies SX6518-NR Manuel

Mellanox Technologies

Mellanox Technologies InfiniScale MTS3600Q-1BNC Manuel utilisateur

Mellanox Technologies

Mellanox Technologies Switch-IB Manuel utilisateur

Mellanox Technologies

Mellanox Technologies SwitchX-2 MSX6018F-1SFS Instructions d'utilisation

Mellanox Technologies

Mellanox Technologies MIS5600Q-10DNC Manuel utilisateur

Mellanox Technologies

Mellanox Technologies Mellanox SX1018 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies Mellanox SX1018 Manuel d'utilisation

Mellanox Technologies

Mellanox Technologies Vantage 8500 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies SwitchX-2 MSX1012B-1BFS Manuel utilisateur

Mellanox Technologies

Mellanox Technologies MIS5025 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies M2401G Manuel utilisateur

Mellanox Technologies

Mellanox Technologies MIS5300 Manuel

Mellanox Technologies

Mellanox Technologies SB7700 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies 1G Switch Air Duct Rail Kit Manuel utilisateur

Mellanox Technologies

Mellanox Technologies InfiniScale III Manuel utilisateur

Manuels Changer populaires d'autres marques

URC

URC Total Control MFS-8 Manuel utilisateur

Azbil

Azbil HPQ-D11 Manuel utilisateur

Sony

Sony DFS-900M Manuel utilisateur

SICK

SICK i17S Manuel utilisateur

F&F

F&F SZR-277 Manuel utilisateur

Shure

Shure UAMS/BK Manuel utilisateur

Dräger

Dräger X-zone Switch Off Manuel utilisateur

Asco

Asco Joucomatic 349 Series Manuel utilisateur

BT Mini Hub Manuel utilisateur

D-Link

D-Link DES-1226G Manuel utilisateur

Johnson Controls

Johnson Controls FS80-C Manuel utilisateur

Alcatel-Lucent

Alcatel-Lucent 7450 ESS-12 Manuel utilisateur

MicroNet

MicroNet SP1200A Manuel utilisateur

Ruijie

Ruijie RG-PF2920 Series Manuel d'installation et d'exploitation

Altinex

Altinex DA1914SX Manuel utilisateur

MiLAN

MiLAN MIL-SM2401M Manuel utilisateur

RJM

RJM MASTERMIND PBC/6X Manuel utilisateur

Atlas

Atlas MMK-KVM8 Manuel utilisateur