1、NetAppxClusteredDataONTAP8.2SANExpressSetupGuidefor32xxSystemsNetApp, Inc.495 East Java Drive Sunnyvale, CA 94089 U.S.Part number: 215-07999-A0May 2013Telephone:+1(408)822-60Fax:+1(408)822-4501Supporttelephone:+1(888)463-8277Web:Feedback:doccommentsTableofContents3ContentsDecidingwhethertousethisReq
2、uirednetworkinformation6Unpackingandmountingcomponents10Locatingthediskshelvesforeachcontroller10Componentmountingpositions11Cablingthe32system13Labelingthecables14Cablingthediskshelfalternatecontrolpathports14CablingthediskshelfSASdataports15CablingtheHApairports22Cablingtheconsoleports22Cablingthe
3、managementports23CablingtheprivateclusterinterconnectforNetAppCN1610switches24CablingtheprivateclusterISLsforNetAppCN1610switches26CablingtheEthernetdatanetworks26CablingtheFibreChanneldatafabrics27Cablingthepowersupplies28Settingupthecluster29Creatingtheclusteronthefirstnode30Joininganodetotheclust
4、er31Troubleshootingbootfailureofanode32Configuringclusterhighavailabilityinatwo-nodecluster33Synchronizingthesystemtimeacrossthecluster34Reducingrootvolumesizetopreventwarningmessages35ConfiguringAutoSupport37Configuringanodefortroubleshootingandmaintenance38Verifyingcabling40Downloadingandinstallin
5、gSystemManager41ProvisioningtestLUNsforverifyingclustersetup42Creatingthetestaggregate42CreatingthetestVserver4344CreatingthetestLUNsSettinguptesthostsandverifyingsystemopr,aTio.464747VerifyingyourhostconfigurationInstallingthehostutilitiesInstallingorenablingMPlOConnectinghoststotargets48Discoverin
6、gLUNsandformattingdisks48Verifyingsetupbywritingandreadingdata49Testingstoragefailover49Deletingthetestprovisioning50Wheretogonext52Copyrightinformation53Trademarkinformation54Howtosendyourcomments55Index56DeCidingWhethertoUSethisguideThisguidedescribeshowtocompletetheinitialmounting,cabling,andtypi
7、calconfigurationofaSANconfigurationusingclusteredDataONTAP8.2softwareand32xxcontrollers.ProceduresarebasedontheOnCommandSystemManagerGUI.Theguidecoverscommonlyusedoptionsandprovidesminimalbackgroundinformation.Theexamplesinthisguidedepictasix-nodecluster,butyoucanalsousetheguidetoconfiguretwo-node,f
8、our-node,oreight-nodeclusters.Thisguideisintendedforcustomerswhowanttoinstallastoragesystemclusterthemselves.Itstartswiththecomponentsdeliveredtoyoursiteandendswiththesystemtestedandreadyforstorageprovisioning.YoushouldusethisguideifyouwantastandardconfigurationfollowingNetAppbestpractices,andyoudon
9、otwantalotofconceptualbackgroundinformationforthetasks.ThisguideisforswitchedclustersusingNetAppclusterswitches.Itdoesnotcovertwo-nodcswitchlessclustersorsingle-nodeclusters.ThisguideassumesthatallnodesintheclusterarcthesamehardwaremodelanduseSAS-connecteddiskshelves.Thisguideassumesanewinstallation
10、andnotanupgradefromapreviousversionofDataONTAPsoftware.ThisguideassumesthatyoudownloadandrunOnCommandSystemManager2.2orlaterforallapplicabletasks.ThisguidedoesnotincludeproceduresusingtheDataONTAPCLIexceptwhentheCLIistheonlywaytocompleteatask.Iftheseassumptionsarcnotcorrectforyourinstallation,orifyo
11、uwantmoreconceptualbackgroundinformation,youshouldrefertothefollowingdocumentationinstead:InstallationandSetupInstructionsforyourhardwareplatfbnmodelUniversalSASandACPCablingGuideDataONTAPSoftwareSetupGuideforCluster-Mode(fornewsystems) ClusteredDataONTAPUpgradeandRevertZDowngradeGuide(forupgrades),
12、ClusteredDataONTAPSANAdministrationGuide ClusteredDataONTAPSwitchSetupGuideforCiscoSwitchesOnCommandSystemManagerOnlineHelp(availablebothfromwithintheproductandasaPDFdownload)ThisdocumentationisavailablefromtheProductDocumentationsectionoftheNetAppSupportSite.RelatedinformationDocumentationontheNetA
13、ppSupportSite:RequirednetworkinformationYoumustgetIPaddressesandothernetworkinformationfromyournetworkadministratorbeforeyoubeginyourSANconfiguration.ManagementIPaddressesandhostnamesYouneedamanagementIPaddressandhostnameforeachnodeandeachclusterswitch.TypeofinformationYourvaluesClusterClusternameHo
14、meEthernetportIPaddressNetmaskNodeIHostnameEthernetporteOMIPaddressNetmaskNode2HostnameEthernetporteOMIPaddressNetmaskNode3HostnameEthernetportcOMIPaddressNetmaskNode4HostnameEthernetporteOMIPaddressNetmaskTypeofinformationYourvaluesNode5HostnameEthernetportCOMIPaddressNetmaskNode6HostnameEthernetpo
15、rteOMIPaddressNetmaskNode7HostnameEthernetporteOMIPaddressNetmaskNode8HostnameEthernetporteOMIPaddressNetmaskClusterswitch1HostnameIPaddressNetmaskClusterswitch2HostnameIPaddressNetmaskiSCSIIPaddressesIfyouplantouseiSCSIforthetestconfiguration,youneedIPaddressesfortheEthernetportsusedforiSCSI.Yousho
16、uldhaveatleasttwoportspernodeforredundancy.Theseaddressesareforasingletestvirtualstorageserver(Vserver).EachproductionVserverusingiSCSIhasitsownsetofIPaddresses.iSCSIportIPaddressNetmaskNode1Porte_Node1Porte_Node1(optional)Porte_Node1(optional)Porte_Node2Porte_Node2Porte_Node2(optional)Porte_Node2(o
17、ptional)Porte_Node3Porte_Node3Porte_Node3(optional)Porte_Node3(optional)Porte_Node4Porte_Node4Porte_Node4(optional)Porte_Node4(optional)Porte_Node5Porte_Node5Porte_Node5(optional)Porte_Node5(optional)Porte_Node6Porte_Node6Porte_Node6(optional)Porte_Node6(optional)Porte_Node7Porte_Node7Porte_Node7(op
18、tional)Porte_iSCSIportIPaddressNetmaskNode7(optional)Porte_Node8Porte_Node8Porte_Node8(optional)Porte_Node8(optional)Porte_UnpackingandmountingcomponentsAsyouunpackandmounttheclustercomponents,itisimportanttotrackwhichdiskshelvesgowithwhichstoragecontrollers.Keepingtrackofthisinformationandlocatingt
19、hediskshelveswiththeirstoragecontrollerssimplifiescablingandensuresthateachcontrollerhasitsrootvolumeandDataONTAPsoftware.Thefactorylabelsdiskshelveswiththestoragecontrollertowhichtheyshouldconnect.Note:IfyouorderedtheclusterinNetAppcabinets,thecomponentsarealreadymounted.Thediskshelfcontainingthero
20、otvolumeismountednexttoitsstoragecontroller.1.ocatingthediskshelvesforeachcontrollerBeforemountingcomponents,youmustlocatethediskshelvesthatbelongtoeachcontroller.ThefactoryinstallstherootvolumeandDataONTAPsoftwareononediskshelfforeachcontroller.Bycablingthecorrectdiskshelvestoeachcontroller,youensu
21、rethateachcontrollerhasitsrootvolume.BeforeyoubeginThecomponentsmustnotbeintherackorcabinetsothatyoucanseethelabelsonthetopofthecomponentchassis.AboutthistaskEachdiskshelfhasastickeronthetopofitschassisthatliststheserialnumberofthecontrollertowhichtheshelfconnects.Thestickeralsoliststhestacknumberfo
22、rconfigurationswithmorethanonestackofdiskshelvespercontroller.Steps1. Asyouunpackthediskshelvesandstoragecontrollers,checkthelabelsonthetopofeachchassis.TheSYSTEMSNistheidentifierofthestoragecontrollerandtheShelfChassisSNistheidentifierofthediskshelf.2. Usingthelabelinformation,groupormarkthediskshe
23、lvesandcontrollerssothatyoucancablethediskshelvestothecorrectstoragecontrollersafterthecomponentsaremounted.Thefollowingillustrationisanexampleofthelabelfordiskshelves.I11NetApp(SYSTEMSN:000000111.ooporStack#:XShelfChassisSN:SHU90956000ED27XXXIHiMImIlIlIMlllComponentmountingpositionsMountingthecompo
24、nentsintherackorcabinetintherecommendedpositionssimplifiescablingandserviceaccess.Dependingonthenumberofdiskshelvesandsizeoftherackorcabinet,youmightneedmultipleracksorcabinets.ThegeneralruleistokeepthecomponentsofanHApairtogethertosimplifycabling.Thefollowingtabicshowsatypicalmountingarrangementwit
25、htwodiskshelvespernode.Eachdiskshelfhasalabelonthetopofitschassisthatliststhestoragecontrollertowhichthatdiskshelfconnects.Notethatthelabelisnotvisibleaftermounting.Rack/cabinet1Rack/cabinet2Rack/cabinet3Clusterswitch1Clusterswitch2Diskshelf2forNode1Diskshelf2forNode3Diskshelf2forNode5Diskshelf1forN
26、ode1Diskshelf1forNode3Diskshelf1forNode5Node1controllerNode3controllerNode5controllerNode2controllerNode4controllerNode6controllerDiskshelf1forNode2Diskshelf1forNode4Diskshelf1forNode6Rack/cabinet 3Disk shelf 2 for Node 6Rack/cabinet1Rack/cabinet2Diskshelf2forNode2Diskshelf2forNode413Cablingthe32xxs
27、ystemCorrectlycablingthesystemenablesthecomponentstocommunicatewitheachotherandwiththerestofyourstorageenvironmentinbothnormalandfailureconditions.1. Steps2. 1.abelingthecablesonpage14Youshouldlabelthecablestosimplifyfutureexpansionandtomaketroubleshootingoftheclustereasier.3. Cablingthediskshelfalt
28、ernatecontrolpathportsonpage14ThealternatecontrolpathusestheACPportsonthediskshelvestoconnecttothestoragecontrollers.Havingacontrolpaththatisseparatefromthedatapathincreasesreliabilityandsimplifiestroubleshooting.4. CablingthediskshelfSASdataportsonpage15TheserialattachedSCSI(SAS)portsenableeachcont
29、rollerintheHApairtoaccessitsowndisksandthedisksofitspartnercontroller.5. CablingtheHApairportsonpage22ThecaandcbportsofthetwocontrollersinanHApairenablethepairtomirroritswritecacheandperformHAmonitoring.6. Cablingtheconsoleportsonpage22Theconsoleportsenableyoutomanagetheserverwhenanetworkconnectioni
30、snotavailable,forexample,duringsoftwareinstallationornetworktroubleshooting.7. Cablingthemanagementportsonpage23ThestoragecontrollermanagementportsenableyoutomanagetheclusterusingtheCLIorsoftwaresuchasSystemManager.Theprivateclusterswitchmanagementportsenableyoutomanagetheswitches.8. Cablingthepriva
31、teclusterinterconnectforNetAppCN1610switchesonpage24Theprivateclusterinterconnectenablestheclusternodestocommunicatewitheachotherandtomovedatawithinthecluster.Theprivateclusterinterconnectusesdedicated10GbEswitches.9. CablingtheprivateclusterISLsforNetAppCN1610switchesonpage26Thefourinter-switchlink
32、s(ISLs)betweenthetwoprivateclusterinterconnectswitchesenablehighavailabilityoftheclusternetwork.10. CablingtheEthernetdatanetworksonpage26TheEthernetportsonthestoragecontrollersenableiSCSIinitiatorsinyourhoststoconnecttotheiSCSItargetsinthestoragecontrollerusingcablesthatyousupply.Theseportscanalsos
33、upportotherEthernetdatatrafficsuchasNFSorCIFS.11. CablingtheFibreChanneldatafabricsonpage27TheFCportsonthestoragecontrollersenableFCinitiatorsinyourhoststoconnecttotheFCtargetsinthestoragecontrollerusingcablesthatyousupply.TheFCportscanalsobeusedtoconnecttobackupdevicessuchastapedrives.12. Cablingth
34、epowersuppliesonpage28Thepowersuppliesprovidetheelectricalpowerneededtooperatethesystem.Thestoragecomponentsshouldconnecttotwoindependentpowersources.1.abelingthecablesYoushouldlabelthecablestosimplifyfutureexpansionandtomaketroubleshootingoftheclustereasier.BeforeyoubeginYouneedthebinderoflabelsshi
35、ppedintheaccessoriesboxforthecluster.AboutthistaskYoucanlabelthecablesatanypointduringtheinstallationprocess.Steps1. Usingthelabelsinthebindersuppliedwiththecluster,labeleachendofeachcable.Youdonotlabelthepowercables.2. Savethebindercontaininganyremaininglabelsforfutureexpansionofthecluster.Cablingt
36、hediskshelfalternatecontrolpathportsThealternatecontrolpathusestheACPportsonthediskshelvestoconnecttothestoragecontrollers.Havingacontrolpaththatisseparatefromthedatapathincreasesreliabilityandsimplifiestroubleshooting.AboutthistaskTheACPcablingusesstandardCAT6Ethernetcables.Steps1. Connecttheprivat
37、emanagementporteOPofcontrollerIinanHApairtotheACPportwiththesquaresymbolonthefirstdiskshelf.TheeOPportislabeledwithawrenchandpadlocksymbol().2. ConnecttheACPportwiththeroundsymboltothenextACPportwithasquaresymbol.Continueinadaisy-chainuntilallACPportsfortheshelvesusedbytheHApairareconnected.3. Conne
38、ctthefinalACPportwiththesquaresymboltotheprivatemanagementporteOPofcontroller2intheHApair.4. RepeatfortheotherHApairsinthecluster.ACPcablingexampleCOErdlorConlrolter2CablingthediskshelfSASdataportsTheserialattachedSCSI(SAS)portsenableeachcontrollerintheHApairtoaccessitsowndisksandthedisksofitspartne
39、rcontroller.BeforeyoubeginYoumusthavelocatedandmarkedwhichdiskshelvesgowitheachstoragecontroller.AboutthistaskDiskshelfcablingisalwaysisolatedwithintheHApairthatownsthedisks.DiskshelvesareneverconnectedtootherHApairsinthecluster.QSFPtoQSFPSAScablesareusedtoconnectdiskshelvestogetherandtoconnectdisks
40、helvestotheSASportsonthecontroller.Note:ItispossibletoinserttheQSFPconnectorincorrectly(upsidedown).Theconnectorcanslideinandevenappeartoclickintoplace.However,thelatchcannotengageunlesstheconnectorisinsertedcorrectly.Afterinsertingthecable,pullontheconnectortoensureitislatched.FordiskshelfI/OModule
41、sandcontrolleronboardSASports,thecablesmustbeinsertedwiththebottomoftheconnectorfacingdown,asshowninthefollowingdiagram.Thebottomoftheconnectorisslightlylongerthanthetop,andthelatchreleaseistypicallyonthebottom.ForcontrollerSASportsonadd-incards,theorientationdependsontheslotinwhichtheSAScardisinsta
42、lled.Inserttheconnectorgently,andbesureitlatchessecurely.Iftheconnectordoesnotlatch,rotateit180degreesandtryagain.TheseinstructionsassumetwostacksofdiskshelvesperHApair.YoucansplityourdiskshelvesintoadditionalstackstoincreaseI/Othroughput.AdditionalstacksrequireaddingadditionalSASportstothestorageco
43、ntrollers.SeetheUniversalSASandACPCablingGuideifyouwanttouseadditionalstacksofdiskshelves.Steps1. SetthediskshelfIDnumbersforeachdiskshelf.Note:ThediskshelfIDnumbersmighthavebeensetinthefactory.TheshelfIDsinthefirststackshouldbesetto10,11,12,andsoon,withthefirstshelfIDsetto10.ThesecondstackIDsshould
44、be20,21,22,andsoon.ThethirdstackIDsshouldstartwith30,thefourthstackIDswith40,thefifthstackIDswith50,andthesixthstackIDswith60.Thispatternmakesiteasytokeeptrackofeachstack.IDsmustbeuniquewithinanHApair.a) Temporarilyconnectpowercordstothediskshelfandturnonthepowertothediskshelfb) Removetheleftearcove
45、rfromtheshelf.c) PressandholdtheU-Sh叩edtababovetheLEDs(DS4243)orshelfIDbutton(DS2246)untilthefirstdigitblinks.d) Pressthetaborbuttonuntilthecorrectnumberisdisplayed.e) Repeatstepscanddfortheseconddigit.f) Pressandholdthetaborbuttonuntilthesecondnumberstopsblinking.Bothnumbersblinkandtheoperatordispl
46、aypanelfaultLEDilluminatesinaboutfiveseconds.g) Power-cyclethediskshelftomakethenewdiskshelfIDtakeeffect.2. Startingwiththefirststackofdiskshelves,cabletheI/Omodulesofthestackstoeachotherasfollows.Ifyoursystemshasonlyonediskshelfperstack,skipthisstep.a) ConnecttheIOMASASportwiththeroundsymbolfromthefirstshelftotheIOMASASportwiththesquaresymbolonthesecondshelf.IOMAiseitheronthetoporonthele
