1、智慧燃气项目部署运维手册XX科技股份有限公司编制目录一、文章编写目的5二、前置准备52.1 配置网络ip52.2 编写相关操作脚本91. 批量执行命令脚本92. 批量重命名脚本113. 批量拷贝脚本134. cm_migrate,sh155. format2.sh166. mountDisk.sh167. network,sh178. node,1.ist189. node,txt182.3hostname及hosts配置181.配置每台节点的hostname182.配置节点ip、hostname映射192.4 禁用SE1.inux192.5 关闭防火墙212.6 设置Swappiness21
2、2.7 关闭透明大页面222.8 配置操作系统本地yum源232.9 安装http服务25NAME=eno1.6777736#随机idUUID=6d21f413-619d-4283-9ba2-e1.a724c1.c593DEVICE=eno1.6777736*系统启动的时候网络接口是否有效（ycsno）ONBOOT=yesPEERDNS=yes”PEERROUTES=yes*IPV6_PEERDNS=yesIPV6一PEERRo1.TES=yesIPV6_PRIVACY=no需要修改的地方有：BOOTPROTO=StaticWhcp改为static（修改）0NB00T=yes#开机启用本配置，
3、一般在最后一行（修改）IPADDR=192.168.139.101#静态IP（增加）GATEWAY=I92.168.139.2#默认网关，虚拟机安装的话，通常是2,也就是VMnet8的网关设置（增加）NETMSK=255.255.255.0#子网掩码（增加）DNS1=192.168.139.2ffDNS配置，虚拟机安装的话,DNS就网关就行，多个DNS网址的话再增加（增加）CREATEUSERcm,%,IDENTIFIEDBYpassword；GRANTA1.1.PRIVI1.EGESONera.*TOcmg%,:F1.USHPRIVI1.EGES；createdatabaseamdefau1
4、tcharactersetutf8;CREATEUSER,amQ,%,IDENTIFIEDBYpassword;GRANTA1.1.PRIVI1.EGESONam.*TOam,0%,;F1.USHPRIVI1.EGES；createdatabasermdefau1.tcharactersetutf8;CREATEUSERrmQ,%,IDENTIFIEDBYpassword：GRANTA1.1.PRIVI1.EGESONrm.*TOrm,%,;F1.USHPRIVI1.EGES；createdatabasehuedefau1.tcharactersetutf8;CREATEUSERhue1%I
5、DENTIFIEDBYpassword;GRANTA1.1.PRIVI1.EGESONhue.*TOhueQ%;F1.USHPRIVI1.EGES；createdatabaseooziedefau1.tcharactersetutf8;CREATEUSER,oozie,%,IDENTIFIEDBYpassword；GRANTA1.1.PRIVI1.EGESONoozie.*TOoozie%;F1.USHPRIVI1.EGES；tota1.940rwxrwxr-x.1rootroot960372Feb108:31mysq1.-connector-java-5.1.34.jarInvxrwxrwx
6、1rootroot31Feb200:52mysq1.-conncctor-java.jar-mysq!-connector-java-5.1.34.jar三、C1.ouderaManager安装3.1配置本地CM源下载CM516.1的安装包，地址为：http：/archive,http：/archive,http：/archive,http：/archive,http：/archive,rpmhttp：/archive,cm5rec1.hat7x8664cm5.16.1/RPMS/x86_64/jdk-6u31-1inux-amd64.rpusrjavajdk1.7.067-cIoudcraV
7、erifyingthatwecanwritetoetcc1oudera-scm-serverCreatingSCMConfigurationfi1.einetcc1.oudera-scm-serverExecuting：usrjavajdk1.7.O67-c1.oudcra/bin/java-cpusrshare/java/mysq1-connector-java,jar：usrsharejavaorac1.e-connector-java.jar：usrsharejavapostgresq1.connector-java,jarusrsharecmf/schema/.1.ib*com.cIo
8、udera.enterprise,dbuti1.DbCommandExecutoretcc1oudera-scm-server/db.propertiescom.cIoudcra.cmf.db.mainDbCommandExecutorINFOSuccessfu1.Iyconnectedtodatabase.A1.Idone,yourSCMdatabaseisconfiguredcorrect1.y!6.2版本命令/opt/cIoudera/cmschemascm_prepare_database.shmysq1.cmcmpassword启动CIOUderaManagerServerSyste
9、mc1.1.startC1.OUdera-SCm-SerVer检查端口是否监听netstat-Inptgrep7180错误：无须任何处理C一f1.1.出现以上情况,使用以下命令yum-yinsta1.1net-too1.s通过http:192.168.139.101:7180/Cmf/1Ogin访问CM四、CDH安装1.1 CDH集群安装向导1 .admin/admin登录到CM2 .同意IiCCnSC协议，点击继续3 .选择60试用，点击继续的o4 .点击“继续”Umno5 .输入主机ip或者名称，点击搜索找到主机后点击继续DODOO6 .安装成功后进入CM管理界面主页五、HBASE安装5.
10、1 准备工作：不同机器之间的时间同步要求每个节点子在30秒root0hadoopNode5#yum-yinsta1.1.ntp#安装ntp软件ErootQhadoopNodeo#指定与洞里云时间同步服务5.2 安装：1 .下栽hbase1.3.22 .解压ambowS1.hadoopNode1、_$tar-xvzfVsoft/hbase-1.3.2bin.tar.gz-Capp3 .配Jt环境变量/.baSe-PrOfiIeHBASEjOMEPATH4 .hbase-env.sh配置java_h(e和ZKambowhadoopNode1Conf$vi$HBASEHOME/conf/hbase-
11、env.shexportJAVAHOME=homcw1.appjdk1.8.O121exportHAD00P_H0ME=/home/w1./app/hadoop-2.7.3exportHBASEMANAGESZK=fa1.sc#禁用Hbasc使用内置zookeperexportHBASE_BACK1.JP_MASTERS=SHBASE-HOME)confbackup-masters#配置HA的第二个节HMaster节点新建一个$HBASE_HSIE/COnf/backup-masters文件viSHBASEJIOME/conf/backup-masters把备用的HMaStCr节点添加：had
12、oopNodc25 .hbase-site.xm1.配置参数homeambowzkdatahdata6 .配Jtregionserver（配置每一个机器名子节点名不要配主节名）在hbaseCOnf/下新建regionserver文件，添加如入内容hadoopNodc3hadoopNode4hadoopNode57 .scp-rhbase到其他节点amboWWhadOOPNode1.confSscp-rapphbase-1. 3.2ambowhadoopNode5：appambowhadoopNodc1confSscp-rapphbasc-1.3.2ambowhadoopNode4：vappam
13、bowhadoopNode1.confSscprapphbase-1.3.2ambowhadoopNodc3：appambHr0hadoopNode1confSscp-rapphbase-1.3.2ambowhadoopNode2：Vapp/ambowhadoopNode1.confSscp/.bashprofi1.eambow0hadoopNodc5:amboWWhadoOPNode1.confSscp/.bash_profi1.eambowhadoopNode4：ambowhadoopNodc1confSscp/,bash_profi1eambowhadoopNode3：ambowhado
14、opNode1confJSscp/.bashprofi1.eambowQhadoopNodc2:各节点重新加载：sourceV.bash_profiIe启动hdfsstart-dfs.sh六、F1.ume安装6.1 安装1 .解压tar-ZXVfaPaChc-fIumcT.6.O-bin.tar.gzambowhadoopNode3f1.umeT.6.0$tar-zxvfapache-fIume-1. 6.O-bin.tar.gz-C7app2 .然后进入f1.ume的目录，修改Conf下的f1.umc-cnv.sh,配置JAVAJIOMEexportJVAJIOME=homeambowapp
15、jdk1.8.O1213 .配置.bash-ProfiIe文件exportF1.UME_HOME=/home/ambowapp/fIume-1.6.OexprotPATH=$PATH:$F1.UMEJioME/bin七、Kafka安装：7.1安装I.下载Apachekafka官方：http：/kafka.apache,o1.wn1.oads.htm1.Sca1.a2.11-kafka_2.11-0.10.2.0.tgz(asc,md5)注:当SCaIa用的是2.11那Kafka选择kafka_2.11-0.102.O.10.2才是Kafka的版本Kafka集群安装：1 .安装JDK&配置JAV
16、AIOME2 .安装Zookeeper参照ZOokeCPCr官网搭建一个ZK集群，并启动ZK臬群。3 .解压Kafka安装包ambowhadoopNode1ambowStar-zxvfkafka_2.11一0.10.2.1.tgz-CVapp/4.配置环境变量exportKKAH()ME-homcambowapp/kafka2.11-0.10.2.1exportPTH=PTH:$KAFKAK)ME/bin5.修改配置文件config/server.propertiesviserver,properties#为依次增长的：0、1、2、3、4,集群中节点唯一idbroker,id-0#删除主题的配
17、置，默认是fa1.se生产环境设为fa1.sede1.ete,topic,enab1.e=true#监听的主机及端口号各节点改为本机相应的hostName1isteners=P1.AINTEXT：/hadoopNode1：9092邓afka的消息数据存储路径1.og.dirs-7homcambowkafka1.)ataIogs#创建主题的时候，默认有1个分区num.partitions=3#指定ZooKeeper集群列表，各节点以逗号分zookeeper,connect-hadoopode1.:2181,hadoopNode2:2181,hadoopNode3:2181,hadoopNode4
18、2181.hadoopNode5:21816 .分发至各个节点ambowhadoopNode1app$scp-rkafka2.11-0.10.2.1ambowhadoopNode5：VappambowhadoopNode1app$scp-rkafka2.11-0.10.2.1ambowhadoopNode4：appambowhadoopNode1app$scp-rkafka_2.11-0.10.2.1ambowhadoopNode3：VappambowhadoopNode1app$scp-rkafka2.11-0.10.2.1ambowhadoopNode2：VappambowhadoopN
19、ode1app$scp-r7.bash_profi1.eambowMIadoOPNode5：、ambowhadoopNode1app1$scp-r、/.bashprofiIcambowhadoopNodc4:ambowhadoopNode1app：$scp-r.bash_profiIeambowWhadOOPNode3：,、ambowhadoopNode1app：$scp-r.bash_profi1.eambowhadoopNode2：source/.bash_profiIe7 .修改各个节点的配置文件：为依次增长的：0、1、2、3、4,集群中节点唯一idbroker,id-0力监听的主机及端口号各节点改为本机相应的hos1.Name1isteners-P1.AINTEXThadoopNode1：90928 .各台节点上启动Kafka服务ambowhadoopNode1app$kafka-server-start.sh$KAFK.*1.1.OME/config/server.properties&注：要先启动各节点ZOOkeePerZkServ1.et.shstartambowhadoopNodc1app$kafka-SCrVer-Stop.sh
