离线安装Apache Ambari

官方文档：https://docs.cloudera.com/HDPDocuments/Ambari-2.7.0.0/bk_ambari-installation/content/

1. 安装准备

1.1 服务器准备

IP OS hostname Service
192.168.40.250 CentOS Linux release 7.3 master Ambari Server
192.168.40.251 CentOS Linux release 7.3 agent-1 Ambari Agent
192.168.40.252 CentOS Linux release 7.3 agent-2 Ambari Agent
192.168.40.253 CentOS Linux release 7.3 agent-3 Ambari Agent

1.2 系统服务安装

yum install openssh wget unzip gcc python -y

1.3 软件下载

  1. Ambari Repositories

  2. HDP Stack Repositories

  3. JDK 1.8

1.4 所有服务器安装JDK

tar vzxf jdk-8u91-linux-x64.tar.gz -C /opt
ln -snf /opt/jdk1.8.0_91 /opt/java
vim /etc/profile
export JAVA_HOME=/opt/java
export PATH=$PATH:$JAVA_HOME/bin
source /etc/profile

1.5 关闭防火墙

systemctl stop firewalld.service
systemctl disable firewalld.service

1.6 设置机器名

vim /etc/sysconfig/network
修改HOSTNAME
vim /etc/hosts
vim /etc/hostname

验证机器名

hostname
hostname -f

1.7 关闭Selinux和THP

查看Selinu

sestatus

关闭Selinu

vim /etc/sysconfig/selinux
修改SELINUX为disable

查看THP

cat /sys/kernel/mm/transparent_hugepage/defrag
cat /sys/kernel/mm/transparent_hugepage/enabled

关闭THP

vim /etc/rc.d/rc.local

if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
        echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
        echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi

重启服务器

reboot

1.8 设置umask

查看umask

umask

设置umask

echo umask 0022 >> /etc/profile

1.9 设置ulimit

查看ulimit

ulimit -Sn
ulimit -Hn

设置ulimit

ulimit -n 10000

1.10 安装ntp

yum install -y ntp

配置ntp服务器

vim /etc/ntp.conf

启动ntp

systemctl start ntpd.service
systemctl enable ntpd.service

1.11 Master做SSH免密登录

ssh-keygen -t rsa
ssh-copy-id agent-1
ssh-copy-id agent-2
ssh-copy-id agent-3

2. 制作本地仓库

2.1 下载依赖包

yum install yum-utils -y
yum repolist
yum install createrepo -y
yum install httpd -y

2.2 解压安装包

mkdir /var/www/html/ambari
mkdir -p /var/www/html/hdp/HDP
mkdir -p /var/www/html/hdp/HDP-GPL-3.1.0.0
mkdir -p /var/www/html/hdp/HDP-UTILS-1.1.0.22
tar -zxvf ambari-2.7.3.0-centos7.tar.gz -C /var/www/html/ambari/
tar -zxvf HDP-3.1.0.0-centos7-rpm.tar.gz -C /var/www/html/hdp/HDP/
tar -zxvf HDP-GPL-3.1.0.0-centos7-rpm.tar.gz -C /var/www/html/hdp/HDP-GPL-3.1.0.0
tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/hdp/HDP-UTILS-1.1.0.22/

2.3 启动httpd服务

systemctl start httpd
systemctl enable httpd
  • 浏览器访问目录空白，请删除目录下的index.html

2.4 配置Ambari Repo

vim /etc/yum.repos.d/ambari.repo

#VERSION_NUMBER=2.7.3.0-139
[ambari-2.7.3.0]
#json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json
name=ambari Version - ambari-2.7.3.0
baseurl=http://192.168.40.250/ambari/centos7/2.7.3.0-139
gpgcheck=1
gpgkey=http://192.168.40.250/ambari/centos7/2.7.3.0-139/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

2.5 配置HDP Repo

vim /etc/yum.repos.d/hdp.repo

#VERSION_NUMBER=3.1.0.0-78
[HDP-3.1.0.0]
name=HDP Version - HDP-3.1.0.0
baseurl=http://192.168.40.250/hdp/HDP/centos7
gpgcheck=1
gpgkey=http://192.168.40.250/hdp/HDP/centos7/3.1.0.0-78/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

2.6 配置HDP-GPL Repo

vim /etc/yum.repos.d/hdp-gpl.repo

#VERSION_NUMBER=3.1.0.0-78
[HDP-GPL-3.1.0.0]
name=HDP-GPL Version - HDP-GPL-3.1.0.0
baseurl=http://192.168.40.250/hdp/HDP-GPL-3.1.0.0/HDP-GPL/centos7
gpgcheck=1
gpgkey=http://192.168.40.250/hdp/HDP-GPL-3.1.0.0/HDP-GPL/centos7/3.1.0.0-78/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

2.7 配置HDP-UTILS Repo

vim /etc/yum.repo.d/hdp-utils.repo

[HDP-UTILS-1.1.0.22]
name=HDP-UTILS Version - HDP-UTILS-1.1.0.22
baseurl=http://192.168.40.250/hdp/HDP-UTILS-1.1.0.22/HDP-UTILS/centos7
gpgcheck=1
gpgkey=http://192.168.40.250/hdp/HDP-UTILS-1.1.0.22/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

2.8 分发配置

scp ambari.repo ambari.repo agent-1:$PWD
scp ambari.repo hdp.repo agent-1:$PWD
scp ambari.repo hdp-gpl.repo agent-1:$PWD
scp ambari.repo hdp-utils.repo agent-1:$PWD

scp ambari.repo ambari.repo agent-2:$PWD
scp ambari.repo hdp.repo agent-2:$PWD
scp ambari.repo hdp-gpl.repo agent-2:$PWD
scp ambari.repo hdp-utils.repo agent-2:$PWD

scp ambari.repo ambari.repo agent-3:$PWD
scp ambari.repo hdp.repo agent-3:$PWD
scp ambari.repo hdp-gpl.repo agent-3:$PWD
scp ambari.repo hdp-utils.repo agent-3:$PWD

2.9 生成本地源

createrepo /var/www/html/ambari/centos7/2.7.3.0-139
createrepo /var/www/html/hdp/HDP/centos7/
createrepo /var/www/html/hdp/HDP-GPL-3.1.0.0/
createrepo /var/www/html/hdp/HDP-UTILS-1.1.0.22/

3. 安装MySQL

3.1 卸载CentOS7系统自带mariadb

rpm -qa|grep mariadb
rpm -e --nodeps mariadb-libs-5.5.44-2.el7.centos.x86_64
rm /etc/my.conf

3.2 检查MySQL是否存在

rpm -qa | grep mysql

3.3 创建mysql组和用户

groupadd mysql
useradd -g mysql mysql

3.4 修改mysql用户密码

passwd mysql

3.5 下载MySQL

官方网站：https://dev.mysql.com/downloads/mysql/5.7.html#downloads

3.6 解压安装包

tar -zxvf mysql-5.7.25-linux-glibc2.12-x86_64.tar.gz -C /opt
ln -snf /opt/mysql-5.7.25-linux-glibc2.12-x86_64 /opt/mysql

3.7 更改用户组和用户

mkdir -p /opt/mysql/data
chown -R mysql:mysql /opt/mysql/

3.8 创建配置文件

vim /opt/mysql/my.conf

[mysql]
socket=/var/lib/mysql/mysql.sock
# set mysql client default chararter
default-character-set=utf8

[mysqld]
socket=/var/lib/mysql/mysql.sock
# set mysql server port  
port = 3306
# set mysql install base dir
basedir=/opt/mysql
# set the data store dir
datadir=/opt/mysql/data
# set the number of allow max connnection
max_connections=200
# set server charactre default encoding
character-set-server=utf8
# the storage engine
default-storage-engine=INNODB
lower_case_table_names=1
max_allowed_packet=16M
explicit_defaults_for_timestamp=true

[mysql.server]
user=mysql
basedir=/opt/mysql

3.9 安装MySQL

cd /opt/mysql
bin/mysql_install_db --user=mysql --basedir=/opt/mysql/ --datadir=/opt/mysql/data/
cp ./support-files/mysql.server /etc/init.d/mysqld
chown 777 my.cnf
chmod +x /etc/init.d/mysqld

3.10 设置开机自启动

chkconfig --level 35 mysqld on
chmod +x /etc/rc.d/init.d/mysqld
chkconfig --add mysqld
chkconfig --list mysqld

3.11 启动MySQL

/etc/init.d/mysqld start

3.12 修改全局变量

vim /etc/profile

export MYSQL_HOME=/opt/mysql
export PATH=$PATH:$MYSQL/bin

source /etc/profile

3.13 获得初始密码

cat /root/.mysql_secret

3.14 修改初始密码

mysql -uroot -p
set PASSWORD = PASSWORD('123456');

4. 初始化MySQL

4.1 创建数据库和用户

create database ambari character set utf8;
CREATE USER 'ambari'@'%' IDENTIFIED BY 'ambari';
grant all privileges on ambari.* to ambari@'%' identified by 'ambari';

create database hive character set utf8;
CREATE USER 'hive'@'%' IDENTIFIED BY 'hive';
grant all privileges on hive.* to hive@'%' identified by 'hive';

create database oozie character set utf8;
CREATE USER 'oozie'@'%' IDENTIFIED BY 'oozie';
grant all privileges on oozie.* to oozie@'%' identified by 'oozie';

create database ranger character set utf8;
CREATE USER 'ranger'@'%' IDENTIFIED BY 'ranger';
grant all privileges on ranger.* to ranger@'%' identified by 'ranger';

create database rangerkms character set utf8;
CREATE USER 'rangerkms'@'%' IDENTIFIED BY 'rangerkms';
grant all privileges on rangerkms.* to rangerkms@'%' identified by 'rangerkms';

create database druid character set utf8;
CREATE USER 'druid'@'%' IDENTIFIED BY 'druid';
grant all privileges on druid.* to druid@'%' identified by 'druid';

FLUSH PRIVILEGES;

4.2 下载驱动

yum install mysql-connector-java
ls /usr/share/java

5. master节点安装ambari-server

5.1 安装ambari-server

yum install ambari-server

5.2 设置ambari-server

ambari-server setup

Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):root
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
==============================================================================
Enter choice (1): 2
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /opt/java/
Validating JDK on Ambari Server...done.
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? y
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (1): 3
Hostname (localhost): localhost
Port (3306): 3306
Database name (ambari): ambari 
Username (ambari): ambari
Enter Database Password (bigdata): amabri
Re-enter password: ambari
Configuring ambari database...
Should ambari use existing default jdbc /usr/share/java/mysql-connector-java.jar [y/n] (y)? y
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
ambari-admin-2.7.3.0.139.jar
....
Ambari repo file contains latest json url http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json, updating stacks repoinfos with it...
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

5.3 初始化ambari数据库

mysql -uambari p
use ambari;
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql;

5.4 启动ambari-server

ambari-server start

检查服务

ambari-server status

停止服务

ambari-server stop

6. 安装ambari-agent

yum -y install ambari-agent

7. 安装服务

7.1 登录ambari

访问ambari

http://<your.ambari.server>:8080

使用默认用户名密码登录ambari

ambari/ambari

7.2 启动Ambari集群安装向导程序

7.3 命名集群

为即将创建的集群命名，然后next

7.4 选择版本

  1. 选择版本HDP-3.1
  2. 选择Use Local Repository
  3. OS选择redhat7
  4. 填写本地仓库地址

如果后续安装没有正确配置本地仓库地址，则手动修改元数据库配置，具体操作如下：

mysql -uambari -p
user ambari;
select * from repo_definition;
update repo_definition set base_url='http://192.168.40.250/hdp/HDP/centos7/3.1.0.0-78' where id=25;
update repo_definition set base_url='http://192.168.40.250/hdp/HDP-GPL-3.1.0.0/HDP-GPL/centos7/3.1.0.0-78' where id=51;
update repo_definition set base_url='http://192.168.40.250/hdp/HDP-UTILS-1.1.0.22/HDP-UTILS/centos7/1.1.0.22' where id=26;

7.5 安装选项

  1. 填写agent的机器名，一行写一个
  2. 复制master的私钥，并粘贴

7.6 确认主机

如果确认主机失败，请检查本地仓库地址是否配置成功和NTP服务配置是否正确

7.7 选择服务

选择必要服务和slave节点

7.8 安装服务

自动安装服务

results matching ""

    No results matching ""