Mellanox Technologies MIS5025 Manuel utilisateur

www.mellanox.com

36 Port InfiniBand QDR Switch Platform

Installation Guide

PNs: MIS5025, MIS5030, MIS5035

Rev 1.8

Mellanox Technologies

350 Oakmead Parkway Suite 100

Sunnyvale, CA 94085

U.S.A.

www.mellanox.com

Tel: (408) 970-3400

Fax: (408) 970-3403

Mellanox Technologies, Ltd.

PO Box 586 Hermon Building

Yokneam 20692

Israel

Tel: +972-4-909-7200

Fax: +972-4-959-3245

Mellanox®, BridgeX®, ConnectX®, InfiniBlast®, InfiniBridge®, InfiniHost®, InfiniRISC®,

InfiniScale®, InfiniPCI®, PhyX® and Virtual Protocol Interconnect® are registered trademarks

of Mellanox Technologies, Ltd. CORE-Direct and FabricIT are trademarks of Mellanox

Technologies, Ltd. All other marks and names mentioned herein may be trademarks of their

respective companies.

36 Port InfiniBand QDR Switch Platform Installation Guide

Document Number: 3135

Rev 1.8

Mellanox Technologies

NOTE:

THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED DOCUMENTATION ARE

PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE

OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE

CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX

TECHNOLOGIES TO FULLY QUALIFY THE PRODUCT(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX

TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE

HIGHEST QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED

WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE

DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT,

INDIRECT, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND (INCLUDING, BUT NOT LIMITED TO,

PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS

INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,

OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE PRODUCT(S) AND

RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Rev 1.8

Mellanox Technologies 3

Table of Contents

Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4

List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5

About this Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

Intended Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

Related Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

Document Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

Internally Managed vs. Externally Managed . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9

Installing the Switch in the Rack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

Minimum and Maximum Rack Depth for this Switch . . . . . . . . . . . . . . . . . .11

Installing the Switch in the Rack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12

Grounding the Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19

Configuring the Switch for the First Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20

Unified Fabric Manager (UFM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27

FabricIT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28

Starting an SSH Connection to the Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30

Starting a WebUI Connection to the Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . .31

Resetting the Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

Rev 1.8

Mellanox Technologies

List of Figures

Figure 1: Rack Rail Kit Parts MIS00083/85 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12

Figure 2: Rack Rail Kit Parts MIS000079 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13

Figure 3: Installation Options Available for Short Switches . . . . . . . . . . . . . . . . . . . . . . . .14

Figure 4: Installation Options Available for Standard Length Switches . . . . . . . . . . . . . . .14

Figure 5: For Short Switches Which Side of the Rack Do You Want the Connectors ? . . .15

Figure 6: Making Room for the Power Cord . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16

Figure 7: Screwing on the Bracket . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16

Figure 8: Screwing on the Rail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17

Figure 9: Inserting the Caged Nuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17

Figure 10: Connect Bracket to Rack Vertical support . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18

Figure 11: Externally Managed Management Interface . . . . . . . . . . . . . . . . . . . . . . . . . . .20

Figure 12: License Key Entitlement Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28

Figure 13: License Key Generation Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29

Figure 14: Reset Button . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

Rev 1.8

Mellanox Technologies 5

List of Tables

Table 1: Revision History of this User’s Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6

Table 2: Reference Documents and Web Sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

Table 3: Switch Management by OPN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9

Table 4: Switches and Rail Kits for Various Rack Depths . . . . . . . . . . . . . . . . . . . . . . . . .10

Table 5: Mellanox Part Numbering Legend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10

Table 6: Serial Terminal Program Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21

Table 7: Configuration Wizard Session - IP Configuration by DHCP . . . . . . . . . . . . . . . .21

Table 8: Configuration Wizard Session - IP Zeroconf Configuration . . . . . . . . . . . . . . . . .23

Table 9: Configuration Wizard Session - Static IP Configuration . . . . . . . . . . . . . . . . . . .24

Rev 1.8

Mellanox Technologies

Revision History

Table 1 - Revision History of this User’s Manual

Revision Date Details

1.8 November

2012 • Added Note to See “Installing the Switch in the

Rack” on page 12.regarding lack of support for stan-

dard length switch installation with power side next to

the vertical support

• Added Figure 4 and Figure 5 and text regarding lack

of support for Standard installation with the connector

side deeper in the rack

1.7 April 2012 Fixed disclaimer typo

Fixed TOC hierarchy

1.6 Sept. 2011 Changes to the Reset button section

Removed 5031switch

1.5 May 2011 Added section on UFM

1.4 July 2010 Added files to change this into a book.Added last page

with barcode

1.3 June 2010 Added “Getting the License Key”

1.2 May 2010 Added Note: ETH Ports Must be 10/100M

1.1 April 2010 Minor formatting

1.0 April 2010 Initial Release

Rev 1.8

Mellanox Technologies 7

About this Manual

This manual provides installation and set-up instructions for the IS50XX series QSFP top of rack

InfiniBand Switch Platform.

Intended Audience

This manual is intended for users and system administrators responsible for installing and setting

up the switch platform.

The manual assumes familiarity with the InfiniBand®architecture specification.

Related Documentation

The documentation set accompanying the QSFP Chassis InfiniBand Switch platform includes the

following:

Table 2 - Reference Documents and Web Sites

Document Conventions

When discussing memory sizes, MB and MBytes are used in this document to mean size in mega bytes. The use of

Mb or Mbits (small b) indicates size in mega bits.

Switch Firmware and Firmware Update Tools See 

http://www.mellanox.com/content/

pages.php?pg=management_tools&menu_section=34

Note that the Switch System described in this manual is

based on Mellanox Technologies’ InfiniScale IV switch

device.

FabricIT TM Enterprise Fabric Management

(EFM) Software User’s Manual Talk to your Mellanox representative for information

regarding licensing and implementation of the FabricIT

Enterprise Fabric Management Software System.

See

This symbol indicates a recommendation.

This symbol indicates importatnt information for the user and installer.

This symbol indicates a potential danger of damage to the hardware or software.

Rev 1.8

Mellanox Technologies

This symbol indicates a potential danger of injury to users and / or installers and poten-

tial damage to the hardware or software.

Rev 1.8

Mellanox Technologies 9

Internally Managed vs. Externally Managed

The following table shows which switches come with a management CPU and which do not.

Managed switches can be upgraded from simple chassis management to full fabric management

by purchasing an additional license. For more information on buying a license see the user’s

manual.

Unmanaged switches are plug and play out of the box. All switches come with the latest firm-

ware burned on the Flash. Update the firmware on unmanaged switches in-band only. When new

firmware is available you will receive an email with the link to the Mellanox firmware download

site. The download site has the Mellanox firmware tool package and full instructions for updating

firmware.

All managed switches have internal chassis management and can manage up to 108 nodes.

FabricIT can support IB fabrics of up to 648 nodes. Managed switches need an initial configura-

tion before they will start working. See Table 3 for details.

Should your unmanaged switch have CONSOLE, Ethernet, and USB connectors, they will not

work. Only the I2C connector will work on unmanaged switches.

Table 3 - Switch Management by OPN

Family Managed /

Unmanaged Management Connections

IS5025 Unman-

aged No Management

Plug and play 

All firmware updates should be done in-band using Mellanox Firm-

ware Management Tools. 12C port access using MTUSB-1 device is

required for firmware updates if in-band burning is not possible.

IS5030 Managed RS232 cable DB9 to RJ45 included in the box to connect to host PC

for initial configuration of the switch. After initial configuration the

switch can be managed through the Ethernet using a remote connec-

tion.

This switch is capable of Fabric management for an unlimited num-

ber of nodes, with the purchase of Mellanox UFM.

IS5035 Managed RS232 cable DB9 to RJ45 included in the box to connect to host PC

for initial configuration of the switch. After initial configuration the

switch can be managed through the Ethernet using a remote connec-

tion.

This switch is capable of Fabric management for an unlimited num-

ber of nodes, with the purchase of Mellanox UFM.

Rev 1.8

Mellanox Technologies

Table 4 - Switches and Rail Kits for Various Rack Depths

Switch S(Standard)/ B (Short) Rack Size Rail Kit #

Short 50cm to 80cm MIS00008

Short 38cm to 50cm MIS00007

Standard 50cm to 80cm MIS00008

Table 5 - Mellanox Part Numbering Legend

Place Field Decoder

IS System Type InfiniScale Switch

50 Model Family

FF Form factor 25 = 36 Ports Unmanaged

30 = 36 Ports and Chassis Management with

Fabric

Management up to 108 nodes

35 = 36 ports with chassis management and Fab-

ric

Management up to 2000 nodes

CInfiniBand Port Config. Q= QDR, D= DDR

- Separator

P# of Power Supplies 0=0, 1=1, 2=2....

MDepth of the Unit S = standard depth, B = short depth

YAir Flow direction R= Connector side to PSU side airflow

F= PSU side to Connector side airflow

R RoHS C=RoHS5, X=RoHS6

Ce manuel convient aux modèles suivants

Table des matières

Autres manuels Mellanox Technologies Changer

Mellanox Technologies

Mellanox Technologies Grid Director 4036E Manuel utilisateur

Mellanox Technologies

Mellanox Technologies MIS5024 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies MIS5600Q-10DNC Manuel utilisateur

Mellanox Technologies

Mellanox Technologies QM8700 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies SX1710 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies SX1012 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies SX6710 Instructions d'utilisation

Mellanox Technologies

Mellanox Technologies MIS5035Q-2SFC Manuel utilisateur

Mellanox Technologies

Mellanox Technologies Spectrum SN2700 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies SX1710 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies SX6015 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies IS5025D-1BRC Manuel utilisateur

Mellanox Technologies

Mellanox Technologies TX6000 Instructions d'utilisation

Mellanox Technologies

Mellanox Technologies Grid Director 4700 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies Grid Director 4036 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies Mellanox SX1018 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies SwitchX-2 MSX6018F-1SFS Instructions d'utilisation

Mellanox Technologies

Mellanox Technologies InfiniScale MTS3600Q-1BNC Manuel utilisateur

Mellanox Technologies

Mellanox Technologies MSX6518-6R Manuel

Mellanox Technologies

Mellanox Technologies MTS3600 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies SB7700 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies IS5025 Manuel utilisateur

Mellanox Technologies

Mellanox Technologies InfiniScale MTS3600Q-1BNC Manuel utilisateur

Mellanox Technologies

Mellanox Technologies InfiniScale III M2401G 20Gb/s InfiniBand Manuel utilisateur

Manuels Changer populaires d'autres marques

URC

URC Total Control MFS-8 Manuel utilisateur

Azbil

Azbil HPQ-D11 Manuel utilisateur

Sony

Sony DFS-900M Manuel utilisateur

SICK

SICK i17S Manuel utilisateur

F&F

F&F SZR-277 Manuel utilisateur

Shure

Shure UAMS/BK Manuel utilisateur

Dräger

Dräger X-zone Switch Off Manuel utilisateur

Asco

Asco Joucomatic 349 Series Manuel utilisateur

BT Mini Hub Manuel utilisateur

D-Link

D-Link DES-1226G Manuel utilisateur

Johnson Controls

Johnson Controls FS80-C Manuel utilisateur

Alcatel-Lucent

Alcatel-Lucent 7450 ESS-12 Manuel utilisateur

MicroNet

MicroNet SP1200A Manuel utilisateur

Ruijie

Ruijie RG-PF2920 Series Manuel d'installation et d'exploitation

Altinex

Altinex DA1914SX Manuel utilisateur

MiLAN

MiLAN MIL-SM2401M Manuel utilisateur

RJM

RJM MASTERMIND PBC/6X Manuel utilisateur

Atlas

Atlas MMK-KVM8 Manuel utilisateur