Solaris10与EMCCX310c实现FCSAN实施手册V2.doc

Solaris 10与EMC CX3-10c实现FC-SAN实施手册现场背景：服务器3台，2台Sun Fire V490，1台Sun Fire V250。存储为DELL|EMC CX3-10c，配备8TB硬盘，一台Brocade SW200E光纤交换机。目的：实现FC-SAN构架。拓扑图：实施步骤：1、 先完成Dell|EMC CX3-10c初始化，配置两个管理IP，分别为：XX.XX.XX.XX。2、 建立服务器、光纤交换机、存储之间的链路。3、 初始化光纤交换机，配置管理地址：XX.XX.XX.XX4、 在Navisphere Manager管理界面发现链路。5、 在服务器端安装Navisphere agent注册，安装PowerPath多链路管理软件。具体安装操作如下：l Navisphere agent的安装与配置先将光盘mount上后，再执行下面的命令：pkgadd -d NAVIAGNT.pkg会出现下述提示信息，按照提示信息选择安装即可。Select the packages that you want to install by doing one of thefollowing:? Enter 1 (to install the host agent)or? Enter 2 (to install CLI) or Enter All (to install both the host agentand CLI)Enter y and y to begin the installation.The installation program looks for any agent configuration filesyou may already have on your system. If the program doesnot find any agent configuration files, then you have finishedinstalling the agent; go to step 10q. If the program does find anyexisting configuration files, it displays a message like the following:At Least 1 saved config file exists for NavisphereAgent.Please select 1 of the following:1Restore/etc/Navisphere/.Naviagent-config.000120:10592Restore/etc/Navisphere/.Naviagent-config.000121:14083 QuitSelect number 1 - 3.修改agent.config配置文件，主要方法如下：修改/etc/Navisphere/agent.configVi /etc/Navisphere/agent.config 修改内容为：# Navisphere agent.config# -# This file is used by Navisphere Agent to configure# site specific values.# All entries in this file have this form:# object value-list# Where "object" tells what sort of information is to# be configured, and "value-list" is a# whitespace-separated list of values to be configured.# Samples, which should be edited, appear below for all# permissible types of objects.# White space (tabs, space, line-feeds,# carriage-returns) and blank lines are ignored;# To continue a line onto the next line, use "" as the# last character of that line. The "" is dropped when# the next line is concatenated to the continued line.# Description specification# -# The "clarDescr" object takes arbitrary text values# intended to describe the agent and its location.#clarDescr Navisphere Agent# The "clarContact" object takes arbitrary text values# intended to describe who is responsible for the# configured agent, and how to reach them.#clarContact John Smith, 800-555-1212# Device specification.# -# The format of "device" objects for CLARiiON devices# is:# devicedevice-specname "description"# where the device-specification is the device name to# open to access the CLARiiON device. It can be a# SCSI-2 file name. If this is "auto", all attached# arrays with bound LUNs will be automatically# configured. (Currently, dummy arguments are required# for name and description, when using# auto-configuration.) This can NOT be defaulted.# The name is the per CLARiiON name that can be used to# identify each CLARiiON to the manager application.# The name is REQUIRED and cannot contain spaces (even# if surrounded by quotes). This can NOT be defaulted.# The description is a quoted string that contains a# brief description of the CLARiiON chassis being# managed. It defaults to "" if not specified, and to# the device name for auto-configuration.# SAMPLES:# device c0t0d0 NAVISPHERE-1 "NAVISPHERE-1"# automatically detect manageable devicesdevice auto auto# User Specification.# -# The format of "user" objects is"# user localUserId# OR# user userIdhostname在此处添加存储系统SPA和SPB管理地址即可user systemXX.XX.XX.XX user systemXX.XX.XX.XX# These build a list of users who are allowed to perform# WRITE operations to the CLARiiON devices managed by# this agent. By default, as specified below, only root# on this machine has WRITE access. Others can be# granted access by including their login name and host# which they can access from. A null list prevents# anyone, including admin, from having write access. In# order to get write access, this file would have to be# modified and the Agent stopped and restarted to# re-read the configuration file, or an appropriate# remote configuration file would have to be sent to the# agent from an already-priviledged user.user root # only on this machine#user sbluepicasso # individual user "sblue" on# host "picasso"#user lgreenhannibal # individual user "lgreen" on# host "hannibal"# Poll specification# -# The "poll" object takes a numerical count of seconds.# This specifies how often the agent solicits# information from the array, rather than using cached# information. The minimum is "10".#poll 60# Eventlog specification# -# The eventlog object takes a numerical count of log# entries.# This defines the maximum number of SP log events# received by the agent on startup, and received by# clients per poll. A value of 0 causes the devices# entire log to be read at startup. A minimum of 100# events will be read on serial, and 2048 on fibre,# overriding any lower value configured below.#eventlog 100# Baud Rate specification# -# If the agent is using a serial port to manage a# device, the next entry can specify either 9600 baud# or 19200 baud.#baud 9600# end of configuration file#保存配置文件，然后可以启动/停止agent服务。命令如下：bash3.00#/etc/init.d/agent stopbash3.00#/etc/init.d/agent start如果安装配置正常，启动agent服务后在Navisphere Manager管理界面中的Connective中会看到链路状态register为yes。如register显示为no，可以考虑重启服务器或者重启agent服务，或者在Navisphere Manager界面中进行手动注册也可以。注册完成后会在链路状态中显示主机名称等信息。l PowerPath的安装与注册PowerPath的安装1、Mount CDROM2、bash3.00# mkdir /unload3、bash3.00#cd /unload4、bash3.00#cp <source_location>/<package_name>. 从光盘将安装文件拷贝到新建的目录下面。5、bash3.00# gunzip <package_name> 解压缩文件6、bash3.00# tar -xvf <package_name> 解压缩文件7、bash3.00#pkgadd d . 安装PowerPath按照下述提示操作：SPARC host:The following packages are available:EMCpower EMC PowerPath(sparc) 5.2.x_bxxxSelect package(s) you wish to process (or 'all' toprocess all packages). (default: all) ?,?,q:x86 (i386) host:The following packages are available:EMCpower EMC PowerPath(i386) 5.2.x_bxxxSelect package(s) you wish to process (or 'all' toprocess all packages). (default: all) ?,?,q:Enter 1 and press ENTER. Enter package base directory (default: /opt):Press ENTER to accept the default base directory (/opt), or type thepath to an alternate base directory and press ENTER.This package contains scripts which will be executedwith super-user permission during the process ofinstalling this package.Do you want to continue with the installation of<EMCpower> y,n,?Enter y and press ENTER.Installation of <EMCpower> was successful.8、安装完成后需要进行PowerPath的注册注册方法如下：bash3.00#/etc/emcpreg -installYou see the following output:= EMC PowerPath Registration =Do you have a new registration key or keys to enter? nType y and press ENTER.You see the following output:Enter the registration key(s) for your product(s),one per line, pressing Enter after each key.After typing all keys, press Enter again.Key (Enter if done):Enter the PowerPath registration key and press ENTER. You can alsoadd keys for other technologies, such as PowerPath MigrationEnabler or PowerPath Encryption with RSA.If you entered a valid key, you see the following output:1 key(s) successfully added.Key successfully installed.Key (Enter if done):Press ENTER. You see the following output if you entered one key:1 key(s) successfully registered.在PowerPath软件安装注册完成后重启服务器，输入下述命令可以看到服务器与存在之间的两条链路：bash3.00#powermt display dev=all会显示链路情况。还需要将Solaris主机的MPxIO功能关闭。（主要是由于Solaris自带的MPxIO功能与PowerPath的多路径功能相冲突）具体修改文件如下：修改/kernel/drv/fp.conf，修改内容如下：mpxio-disable="yes"修改/kernel/drv/scsi_vhci.conf，修改内容如下：mpxio-disable="yes"修改完成后，运行如下命令：powermt display dev=all powermt check (Choose "A" for all. Clear the dead paths.) devfsadm C (Scan the SCSI bus.) powercf -q (Rebuild the logical link layer.) powermt config (Bring the device paths into PowerPath control.) powermt save (Save the current configuration.)完成配置保存等工作后PowerPath软件即可正常使用。6、 存储建立RAID Group，Storage Group，将服务器与LUN映射关系对应上。7、 服务器端识别存储，格式化硬盘以及做文件系统。具体操作如下：sh-3.00# formatSearching for disks.doneAVAILABLE DISK SELECTIONS: 0. c0t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424> /pci1d,700000/scsi4/sd0,0 1. c0t8d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424> /pci1d,700000/scsi4/sd8,0 2. c3t0d0 <DGC RAID 5 0326 b761dc00> /pci1d,700000/QLGC,qlc2/sd0,0 3. c3t1d0 <DGC RAID 5 0326 b761dc00> /pci1d,700000/QLGC,qlc2/sd1,0 4. emcpower0a <DGC RAID 5 0326 b761dc00> /pseudo/emcp0Specify disk (enter its number): 4selecting emcpower0adisk formattedFORMAT MENU: disk - select a disk type - select (define) a disk type partition - select (define) a partition table current - describe the current disk format - format and analyze the disk repair - repair a defective sector label - write label to the disk analyze - surface analysis defect - defect list management backup - search for backup labels verify - read and display labels inquiry - show vendor, product and revision volname - set 8-character volume name !<cmd> - execute <cmd>, then return quitformat> pPARTITION MENU: 0 - change 0' partition 1 - change 1' partition 2 - change 2' partition 3 - change 3' partition 4 - change 4' partition 5 - change 5' partition 6 - change 6' partition select - select a predefined table modify - modify a predefined partition table name - name the current table print - display the current table label - write partition map and label to the disk !<cmd> - execute <cmd>, then return quitpartition> pCurrent partition table (original):Total disk sectors available: 3076627422 + 16384 (reserved sectors)Part Tag Flag First Sector Size Last Sector0 root wm 34 128.00MB 262177 1 swap wu 262178 128.00MB 524321 2 unassigned wm 0 0 0 3 unassigned wm 0 0 0 4 unassigned wm 0 0 0 5 unassigned wm 0 0 0 6 usr wm 524322 1.43TB 3076627421 8 reserved wm 3076627422 8.00MB 3076643805 partition> labelReady to label disk, continue? ypartition> qFORMAT MENU: disk - select a disk type - select (define) a disk type partition - select (define) a partition table current - describe the current disk format - format and analyze the disk repair - repair a defective sector label - write label to the disk analyze - surface analysis defect - defect list management backup - search for backup labels verify - read and display labels inquiry - show vendor, product and revision volname - set 8-character volume name !<cmd> - execute <cmd>, then return quitformat> q创建文件系统，Solaris默认格式bash-3.00# newfs /dev/rdsk/emcpower0g newfs: 建构新的文件系统 /dev/rdsk/ emcpower0g: (y/n)？ y警告：最后一个未分配柱面中的 1104 扇区/dev/rdsk/ emcpower0g: 在 500668 磁道上，48 扇区的 128 柱面的 3076103088 扇区1502003.5MB 在 3502 柱面群组 (143 c/g, 429.00MB/g, 448 i/g)超块 (fsck -F ufs -o b=#) 备份在：32, 878752, 1757472, 2636192, 3514912, 4393632, 5272352, 6151072, 7029792, 7908512,初始化柱面组：.最后 10 个柱面组的超级块备份位置： 3068047904, 3068926624, 3069805344, 3070684064, 3071562784, 3072441504, 3073320224, 3074198944, 3075077664, 3075956384,文件系统做完。挂载硬盘bash-3.00# mkdir emcdiskbash-3.00# mount /dev/dsk/ emcpower0g /mnt/emcdisk/其它链路也如上述操作，完成后运行下面命令可以看到磁盘大小以及利用情况。bash-3.00# df -kh文件系统 大小 用了 可用 容量 挂接在/dev/dsk/c0t0d0s0 32G 1.1G 30G 4% /devices 0K 0K 0K 0% /devicesctfs 0K 0K 0K 0% /system/contractproc 0K 0K 0K 0% /procmnttab 0K 0K 0K 0% /etc/mnttabswap 2.1G 1.0M 2.1G 1% /etc/svc/volatileobjfs 0K 0K 0K 0% /system/object/dev/dsk/c0t0d0s3 23G 4.3G 19G 19% /usrfd 0K 0K 0K 0% /dev/fd/dev/dsk/c0t8d0s7 67G 743M 66G 2% /varswap 2.1G 312K 2.1G 1% /tmpswap 2.1G 24K 2.1G 1% /var/run/dev/dsk/c0t0d0s6 1.2G 83M 1.1G 7% /temp/dev/dsk/c0t0d0s7 10G 1.9G 8.2G 19% /export/home/dev/dsk/emcpower0g 1.4T 64M 1.4T 1% /mnt/emcdisk8、 硬盘自动挂载，修改配置文件，配置文件在/etc/vfstab具体修改参考下面文档：bash-3.00# vi /etc/vfstab#device device mount FS fsck mount mount#to mount to fsck point type pass at boot options#fd - /dev/fd fd - no -/proc - /proc proc - no -/dev/dsk/c0t0d0s1 - - swap - no -/dev/dsk/c0t0d0s0 /dev/rdsk/c0t0d0s0 / ufs 1 no -/dev/dsk/c0t0d0s3 /dev/rdsk/c0t0d0s3 /usr ufs 1 no -/dev/dsk/c0t8d0s7 /dev/rdsk/c0t8d0s7 /var ufs 1 no -/dev/dsk/c0t0d0s7 /dev/rdsk/c0t0d0s7 /export/home ufs 2 yes -/dev/dsk/c0t0d0s6 /dev/rdsk/c0t0d0s6 /temp ufs 2 yes -/devices - /devices devfs - no -ctfs - /system/contract ctfs - no -objfs - /system/object objfs - no -swap - /tmp tmpfs - yes -/dev/dsk/emcpower0g /dev/rdsk/emcpower0g /mnt/emcdisk ufs 1 yes -9、 光纤交换机配置zone光纤交换机主要配置管理ip地址，zone等，在初始化时已经完成了ip地址的配置，配置zone的方