1、NetAppxClusteredDataONTAP8.2SANExpressSetupGuidefor62xxSystemsNetApp, Inc.495 East Java Drive Sunnyvale, CA 94089 U.S.Part number: 215-07998-A0May 2013Telephone:+1(408)822-60Fax:+1(408)822-4501Supporttelephone:+1(888)463-8277Web:Feedback:doccommentsTableofContents3ContentsDecidingwhethertousethisReq
2、uirednetworkinformation6Unpackingandmountingcomponents10Locatingthediskshelvesforeachcontroller10Componentmountingpositions11Cablingthe62system13Labelingthecables14Cablingthediskshelfalternatecontrolpathports14CablingthediskshelfSASdataports15CablingtheHApairNVRAMports22Cablingtheconsoleports22Cabli
3、ngthemanagementports23CablingtheprivateclusterinterconnectforNetAppCN1610switches24CablingtheprivateclusterISLsforNetAppCN1610switches26CablingtheEthernetdatanetworks26CablingtheFibreChanneldatafabrics27Cablingthepowersupplies28Settingupthecluster29Creatingtheclusteronthefirstnode30Joininganodetothe
4、cluster31Troubleshootingbootfailureofanode32Configuringclusterhighavailabilityinatwo-nodecluster33Synchronizingthesystemtimeacrossthecluster34Reducingrootvolumesizetopreventwarningmessages35ConfiguringAutoSupport37Configuringanodefortroubleshootingandmaintenance38Verifyingcabling40Downloadingandinst
5、allingSystemManager41ProvisioningtestLUNsforverifyingclustersetup42Creatingthetestaggregate42CreatingthetestVserver4344CreatingthetestLUNsSettinguptesthostsandverifyingsystemopr,aTio.464747VerifyingyourhostconfigurationInstallingthehostutilitiesInstallingorenablingMPlOConnectinghoststotargets48Disco
6、veringLUNsandformattingdisks48Verifyingsetupbywritingandreadingdata49Testingstoragefailover49Deletingthetestprovisioning50Wheretogonext52Copyrightinformation53Trademarkinformation54Howtosendyourcomments55Index56DeCidingWhethertoUSethisguideThisguidedescribeshowtocompletetheinitialmounting,cabling,an
7、dtypicalconfigurationofaSANconfigurationusingclusteredDataONTAP8.2softwareand62xxcontrollers.ProceduresarebasedontheOnCommandSystemManagerGUI.Theguidecoverscommonlyusedoptionsandprovidesminimalbackgroundinformation.Theexamplesinthisguidedepictasix-nodecluster,butyoucanalsousetheguidetoconfiguretwo-n
8、ode,four-node,oreight-nodeclusters.Thisguideisintendedforcustomerswhowanttoinstallastoragesystemclusterthemselves.Itstartswiththecomponentsdeliveredtoyoursiteandendswiththesystemtestedandreadyforstorageprovisioning.YoushouldusethisguideifyouwantastandardconfigurationfollowingNetAppbestpractices,andy
9、oudonotwantalotofconceptualbackgroundinformationforthetasks.ThisguideisforswitchedclustersusingNetAppclusterswitches.Itdoesnotcovertwo-nodcswitchlessclustersorsingle-nodeclusters.ThisguideassumesthatallnodesintheclusterarcthesamehardwaremodelanduseSAS-connecteddiskshelves.Thisguideassumesanewinstall
10、ationandnotanupgradefromapreviousversionofDataONTAPsoftware.ThisguideassumesthatyoudownloadandrunOnCommandSystemManager2.2orlaterforallapplicabletasks.ThisguidedoesnotincludeproceduresusingtheDataONTAPCLIexceptwhentheCLIistheonlywaytocompleteatask.Iftheseassumptionsarcnotcorrectforyourinstallation,o
11、rifyouwantmoreconceptualbackgroundinformation,youshouldrefertothefollowingdocumentationinstead:InstallationandSetupInstructionsforyourhardwareplatfbnmodelUniversalSASandACPCablingGuideDataONTAPSoftwareSetupGuideforCluster-Mode(fornewsystems) ClusteredDataONTAPUpgradeandRevertZDowngradeGuide(forupgra
12、des),ClusteredDataONTAPSANAdministrationGuide ClusteredDataONTAPSwitchSetupGuideforCiscoSwitchesOnCommandSystemManagerOnlineHelp(availablebothfromwithintheproductandasaPDFdownload)ThisdocumentationisavailablefromtheProductDocumentationsectionoftheNetAppSupportSite.RelatedinformationDocumentationonth
13、eNetAppSupportSite:RequirednetworkinformationYoumustgetIPaddressesandothernetworkinformationfromyournetworkadministratorbeforeyoubeginyourSANconfiguration.ManagementIPaddressesandhostnamesYouneedamanagementIPaddressandhostnameforeachnodeandeachclusterswitch.TypeofinformationYourvaluesClusterClustern
14、ameHomeEthernetportIPaddressNetmaskNodeIHostnameEthernetporteOMIPaddressNetmaskNode2HostnameEthernetporteOMIPaddressNetmaskNode3HostnameEthernetportcOMIPaddressNetmaskNode4HostnameEthernetporteOMIPaddressNetmaskTypeofinformationYourvaluesNode5HostnameEthernetportCOMIPaddressNetmaskNode6HostnameEther
15、netporteOMIPaddressNetmaskNode7HostnameEthernetporteOMIPaddressNetmaskNode8HostnameEthernetporteOMIPaddressNetmaskClusterswitch1HostnameIPaddressNetmaskClusterswitch2HostnameIPaddressNetmaskiSCSIIPaddressesIfyouplantouseiSCSIforthetestconfiguration,youneedIPaddressesfortheEthernetportsusedforiSCSI.Y
16、oushouldhaveatleasttwoportspernodeforredundancy.Theseaddressesareforasingletestvirtualstorageserver(Vserver).EachproductionVserverusingiSCSIhasitsownsetofIPaddresses.iSCSIportIPaddressNetmaskNode1Porte_Node1Porte_Node1(optional)Porte_Node1(optional)Porte_Node2Porte_Node2Porte_Node2(optional)Porte_No
17、de2(optional)Porte_Node3Porte_Node3Porte_Node3(optional)Porte_Node3(optional)Porte_Node4Porte_Node4Porte_Node4(optional)Porte_Node4(optional)Porte_Node5Porte_Node5Porte_Node5(optional)Porte_Node5(optional)Porte_Node6Porte_Node6Porte_Node6(optional)Porte_Node6(optional)Porte_Node7Porte_Node7Porte_Nod
18、e7(optional)Porte_iSCSIportIPaddressNetmaskNode7(optional)Porte_Node8Porte_Node8Porte_Node8(optional)Porte_Node8(optional)Porte_UnpackingandmountingcomponentsAsyouunpackandmounttheclustercomponents,itisimportanttotrackwhichdiskshelvesgowithwhichstoragecontrollers.Keepingtrackofthisinformationandloca
19、tingthediskshelveswiththeirstoragecontrollerssimplifiescablingandensuresthateachcontrollerhasitsrootvolumeandDataONTAPsoftware.Thefactorylabelsdiskshelveswiththestoragecontrollertowhichtheyshouldconnect.Note:IfyouorderedtheclusterinNetAppcabinets,thecomponentsarealreadymounted.Thediskshelfcontaining
20、therootvolumeismountednexttoitsstoragecontroller.1.ocatingthediskshelvesforeachcontrollerBeforemountingcomponents,youmustlocatethediskshelvesthatbelongtoeachcontroller.ThefactoryinstallstherootvolumeandDataONTAPsoftwareononediskshelfforeachcontroller.Bycablingthecorrectdiskshelvestoeachcontroller,yo
21、uensurethateachcontrollerhasitsrootvolume.BeforeyoubeginThecomponentsmustnotbeintherackorcabinetsothatyoucanseethelabelsonthetopofthecomponentchassis.AboutthistaskEachdiskshelfhasastickeronthetopofitschassisthatliststheserialnumberofthecontrollertowhichtheshelfconnects.Thestickeralsoliststhestacknum
22、berforconfigurationswithmorethanonestackofdiskshelvespercontroller.Steps1. Asyouunpackthediskshelvesandstoragecontrollers,checkthelabelsonthetopofeachchassis.TheSYSTEMSNistheidentifierofthestoragecontrollerandtheShelfChassisSNistheidentifierofthediskshelf.2. Usingthelabelinformation,groupormarkthedi
23、skshelvesandcontrollerssothatyoucancablethediskshelvestothecorrectstoragecontrollersafterthecomponentsaremounted.Thefollowingillustrationisanexampleofthelabelfordiskshelves.I11NetApp(SYSTEMSN:000000111.ooporStack#:XShelfChassisSN:SHU90956000ED27XXXIHiMImIlIlIMlllComponentmountingpositionsMountingthe
24、componentsintherackorcabinetintherecommendedpositionssimplifiescablingandserviceaccess.Dependingonthenumberofdiskshelvesandsizeoftherackorcabinet,youmightneedmultipleracksorcabinets.ThegeneralruleistokeepthecomponentsofanHApairtogethertosimplifycabling.Thefollowingtabicshowsatypicalmountingarrangeme
25、ntwithtwodiskshelvespernode.A62xxsystemrequirestwotypicalracksorcabinets.Eachdiskshelfhasalabelonthetopofitschassisthatliststhestoragecontrollertowhichthatdiskshelfconnects.Notethatthelabelisnotvisibleaftermounting.Rack/cabinet1Rack/cabinet2Rack/cabinet3Clusterswitch1Clusterswitch2Diskshelf2forNode1
26、Diskshelf2forNode3Diskshelf2forNode5Diskshelf1forNode1Diskshelf1forNode3Diskshelf1forNode5NodeIcontrollerNode3controllerNode5controllerNode2controllerNode4controllerNode6controllerDiskshelf1forNode2Diskshelf1forNode4Diskshelf1forNode6Rack/cabinet 3Disk shelf 2 for Node 6Rack/cabinet1Rack/cabinet2Dis
27、kshelf2forNode2Diskshelf2forNode413Cablingthe62xxsystemCorrectlycablingthesystemenablesthecomponentstocommunicatewitheachotherandwiththerestofyourstorageenvironmentinbothnormalandfailureconditions.1. Steps2. 1.abelingthecablesonpage14Youshouldlabelthecablestosimplifyfutureexpansionandtomaketroublesh
28、ootingoftheclustereasier.3. Cablingthediskshelfalternatecontrolpathportsonpage14ThealternatecontrolpathusestheACPportsonthediskshelvestoconnecttothestoragecontrollers.Havingacontrolpaththatisseparatefromthedatapathincreasesreliabilityandsimplifiestroubleshooting.4. CablingthediskshelfSASdataportsonp
29、age15TheserialattachedSCSI(SAS)portsenableeachcontrollerintheHApairtoaccessitsowndisksandthedisksofitspartnercontroller.5. CablingtheHApairNVRAMportsonpage22ThenonvolatileRAM(NVRAM)portsofthetwocontrollersinanHApairenablethepairtomirroritswritecacheandperformHAmonitoring.6. Cablingtheconsoleportsonp
30、age22Theconsoleportsenableyoutomanagetheserverwhenanetworkconnectionisnotavailable,forexample,duringsoftwareinstallationornetworktroubleshooting.7. Cablingthemanagementportsonpage23ThestoragecontrollermanagementportsenableyoutomanagetheclusterusingtheCLIorsoftwaresuchasSystemManager.Theprivatecluste
31、rswitchmanagementportsenableyoutomanagetheswitches.8. CablingtheprivateclusterinterconnectforNetAppCN1610switchesonpage24Theprivateclusterinterconnectenablestheclusternodestocommunicatewitheachotherandtomovedatawithinthecluster.Theprivateclusterinterconnectusesdedicated10GbEswitches.9. Cablingthepri
32、vateclusterISLsforNetAppCN1610switchesonpage26Thefourinter-switchlinks(ISLs)betweenthetwoprivateclusterinterconnectswitchesenablehighavailabilityoftheclusternetwork.10. CablingtheEthernetdatanetworksonpage26TheEthernetportsonthestoragecontrollersenableiSCSIinitiatorsinyourhoststoconnecttotheiSCSItar
33、getsinthestoragecontrollerusingcablesthatyousupply.TheseportscanalsosupportotherEthernetdatatrafficsuchasNFSorCIFS.11. CablingtheFibreChanneldatafabricsonpage27TheFCportsonthestoragecontrollersenableFCinitiatorsinyourhoststoconnecttotheFCtargetsinthestoragecontrollerusingcablesthatyousupply.TheFCpor
34、tscanalsobeusedtoconnecttobackupdevicessuchastapedrives.12. Cablingthepowersuppliesonpage28Thepowersuppliesprovidetheelectricalpowerneededtooperatethesystem.Thestoragecomponentsshouldconnecttotwoindependentpowersources.1.abelingthecablesYoushouldlabelthecablestosimplifyfutureexpansionandtomaketroubl
35、eshootingoftheclustereasier.BeforeyoubeginYouneedthebinderoflabelsshippedintheaccessoriesboxforthecluster.AboutthistaskYoucanlabelthecablesatanypointduringtheinstallationprocess.Steps1. Usingthelabelsinthebindersuppliedwiththecluster,labeleachendofeachcable.Youdonotlabelthepowercables.2. Savethebind
36、ercontaininganyremaininglabelsforfutureexpansionofthecluster.CablingthediskshelfalternatecontrolpathportsThealternatecontrolpathusestheACPportsonthediskshelvestoconnecttothestoragecontrollers.Havingacontrolpaththatisseparatefromthedatapathincreasesreliabilityandsimplifiestroubleshooting.Aboutthistas
37、kTheACPcablingusesstandardCAT6Ethernetcables.Steps1. ConnecttheprivatemanagementporteOPofcontrollerIinanHApairtotheACPportwiththesquaresymbolonthefirstdiskshelf.TheeOPportislabeledwithawrenchandpadlocksymbol().2. ConnecttheACPportwiththeroundsymboltothenextACPportwithasquaresymbol.Continueinadaisy-c
38、hainuntilallACPportsfortheshelvesusedbytheHApairareconnected.3. ConnectthefinalACPportwiththesquaresymboltotheprivatemanagementporteOPofcontroller2intheHApair.4. RepeatfortheotherHApairsinthecluster.ACPcablingexampleCOErdlorConlrolter2CablingthediskshelfSASdataportsTheserialattachedSCSI(SAS)portsena
39、bleeachcontrollerintheHApairtoaccessitsowndisksandthedisksofitspartnercontroller.BeforeyoubeginYoumusthavelocatedandmarkedwhichdiskshelvesgowitheachstoragecontroller.AboutthistaskDiskshelfcablingisalwaysisolatedwithintheHApairthatownsthedisks.DiskshelvesareneverconnectedtootherHApairsinthecluster.QS
40、FPtoQSFPSAScablesareusedtoconnectdiskshelvestogetherandtoconnectdiskshelvestotheSASportsonthecontroller.Note:ItispossibletoinserttheQSFPconnectorincorrectly(upsidedown).Theconnectorcanslideinandevenappeartoclickintoplace.However,thelatchcannotengageunlesstheconnectorisinsertedcorrectly.Afterinsertin
41、gthecable,pullontheconnectortoensureitislatched.FordiskshelfI/OModulesandcontrolleronboardSASports,thecablesmustbeinsertedwiththebottomoftheconnectorfacingdown,asshowninthefollowingdiagram.Thebottomoftheconnectorisslightlylongerthanthetop,andthelatchreleaseistypicallyonthebottom.ForcontrollerSASport
42、sonadd-incards,theorientationdependsontheslotinwhichtheSAScardisinstalled.Inserttheconnectorgently,andbesureitlatchessecurely.Iftheconnectordoesnotlatch,rotateit180degreesandtryagain.TheseinstructionsassumetwostacksofdiskshelvesperHApair.YoucansplityourdiskshelvesintoadditionalstackstoincreaseI/Othr
43、oughput.AdditionalstacksrequireaddingadditionalSASportstothestoragecontrollers.SeetheUniversalSASandACPCablingGuideifyouwanttouseadditionalstacksofdiskshelves.Steps1. SetthediskshelfIDnumbersforeachdiskshelf.Note:ThediskshelfIDnumbersmighthavebeensetinthefactory.TheshelfIDsinthefirststackshouldbeset
44、to10,11,12,andsoon,withthefirstshelfIDsetto10.ThesecondstackIDsshouldbe20,21,22,andsoon.ThethirdstackIDsshouldstartwith30,thefourthstackIDswith40,thefifthstackIDswith50,andthesixthstackIDswith60.Thispatternmakesiteasytokeeptrackofeachstack.IDsmustbeuniquewithinanHApair.a) Temporarilyconnectpowercord
45、stothediskshelfandturnonthepowertothediskshelfb) Removetheleftearcoverfromtheshelf.c) PressandholdtheU-Sh叩edtababovetheLEDs(DS4243)orshelfIDbutton(DS2246)untilthefirstdigitblinks.d) Pressthetaborbuttonuntilthecorrectnumberisdisplayed.e) Repeatstepscanddfortheseconddigit.f) Pressandholdthetaborbutton
46、untilthesecondnumberstopsblinking.BothnumbersblinkandtheoperatordisplaypanelfaultLEDilluminatesinaboutfiveseconds.g) Power-cyclethediskshelftomakethenewdiskshelfIDtakeeffect.2. Startingwiththefirststackofdiskshelves,cabletheI/Omodulesofthestackstoeachotherasfollows.Ifyoursystemshasonlyonediskshelfperstack,skipthisstep.a) ConnecttheIOMASASportwiththeroundsymbolfromthefirstshelftotheIOMASASport
