一个80后
程序员的笔记

Docker配置redis集群启动哨兵模式

bbhsky阅读(98)

使用Docker在三台服务器上安装部署了1主2从3哨兵

192.168.140.197 master
192.168.140.198 slave
192.168.140.199 slave

安装Docker环境

创建配置文件存放目录

mkdir -p /data/disk/redis/data

下载Redis配置文件

cd /data/disk/redis/
wget http://download.redis.io/redis-stable/redis.conf

修改Redis.conf文件

# 修改bind  或者直接注释掉
bind 0.0.0.0
# 保护模式 配合bindip或者设置密码等访问 
protected-mode no
# redis密码
requirepass 123456
# redis日志地址
logfile "/data/redis.log"
# 主节点密码 建议与主节点保值一致,否则故障切换时密码错误导致不同步
masterauth 123456

# 在slave的节点配置上添加 master上不需要
# 主节点IP 主节点端口地址
replicaof 192.168.140.197 6379

创建并启动节点

docker run -d --restart=always --privileged=true -p 6379:6379 \
           -v /data/disk/redis/redis.conf:/etc/redis/redis.conf \
           -v /data/disk/redis/data:/data \
           --name redis redis:6.2.4 \
           redis-server /etc/redis/redis.conf \
           --appendonly yes

查看Redis状态

# 进入容器连接 redis客户端
docker exec -it redis redis-cli
# 输入密码
auth 123456
# 查看redis状态
info replication
# Master节点状态
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=192.168.140.199,port=6379,state=online,offset=1137958,lag=1
slave1:ip=192.168.140.198,port=6379,state=online,offset=1138098,lag=0
master_failover_state:no-failover
master_replid:8bbf12424cb844c55ef81eb0f677a68feef15ebf
master_replid2:9385ed583edd049d68454b2b42b05e7efe853594
master_repl_offset:1138098
second_repl_offset:1118607
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:364096
repl_backlog_histlen:774003
# Slave节点状态
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:192.168.140.197
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:1154590
slave_priority:100
slave_read_only:1
replica_announced:1
connected_slaves:0
master_failover_state:no-failover
master_replid:8bbf12424cb844c55ef81eb0f677a68feef15ebf
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:1154590
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1137236
repl_backlog_histlen:17355

# 如果从节点 master_link_status的值为down 查看一下redis.conf中设置的密码对不对
# 主节点requirepass与从节点的masterauth是否一样,不一样修改一样重新启动

验证是否同步

# Master节点
set bbhsky 123456
# Slave节点
get bbhsky
# 如果能正确获取到 证明搭建没问题了

哨兵模式Sentinel部署

哨兵节点建议为奇数,我们在三个redis服务器上分别部署(建议:单独一台、或多台服务器部署)

创建配置目录

mkdir /data/disk/redis-sentinel/data

下载并修改配置文件



cd /data/disk/redis-sentinel/
wget http://download.redis.io/redis-stable/sentinel.conf

# Reids Sentinel日志地址
logfile "/data/sentinel.log"
# mymaster 可以修改为自己的名称 但是配置内所有名称都需要改掉
# Master 节点的地址 和端口   后面的2 代表有2个sentinel认为主节点失败时开始选举新的主节点
sentinel monitor mymaster 192.168.140.197 6379 2

创建并启动节点

# 如果是在一台机器上启动3个哨兵,需要修改端口  例如:-p 26379:26379  -p 36379:26379 -p 46379:26379,这时候容器名称也需要改变;
docker run -it --restart=unless-stopped --privileged=true \
           --name redis-sentinel -p 26379:26379 \
           -v /data/disk/redis-sentinel/data:/data \
           -v /data/disk/redis-sentinel/sentinel.conf:/usr/local/etc/redis/sentinel.conf \
           -d redis \
           redis-sentinel /usr/local/etc/redis/sentinel.conf

查看哨兵监控情况

# 进入容器连接 redis客户端
docker exec -it redis-sentinel redis-cli -p 26379
# 监控哨兵情况
127.0.0.1:26379> sentinel master mymaster
 1) "name"
 2) "mymaster"
 3) "ip"
 4) "192.168.140.197"
 5) "port"
 6) "6379"
 7) "runid"
 8) "7306162d9f6446b8f9baa91d1fd65064fba86cd6"
 9) "flags"
10) "master"
11) "link-pending-commands"
12) "0"
13) "link-refcount"
14) "1"
15) "last-ping-sent"
16) "0"
17) "last-ok-ping-reply"
18) "860"
19) "last-ping-reply"
20) "860"
21) "down-after-milliseconds"
22) "30000"
23) "info-refresh"
24) "38"
25) "role-reported"
26) "master"
27) "role-reported-time"
28) "1115196"
29) "config-epoch"
30) "3"
31) "num-slaves"
32) "2"
33) "num-other-sentinels"
34) "3"
35) "quorum"
36) "2"
37) "failover-timeout"
38) "180000"
39) "parallel-syncs"
40) "1"
# name 监控名称
# ip 主节点IP
# port 主节点端口
# num-slaves 当前有几个从节点
# num-other-sentinels 有几个哨兵
# quorum  有几个哨兵失败重新选举
# failover-timeout 切换时间 秒

rancher解决跨域问题

bbhsky阅读(1303)

解决方法:

与添加大小的方法一样
nginx.ingress.kubernetes.io/cors-allow-headers: DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization
nginx.ingress.kubernetes.io/cors-allow-methods: PUT, GET, POST, OPTIONS
nginx.ingress.kubernetes.io/cors-allow-origin: '*'
nginx.ingress.kubernetes.io/enable-cors: "true"

参考资料:https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#enable-cors

安装kubectl

bbhsky阅读(431)

1、下载最新版的kubectl:

curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl

下载指定版本的kubectl需要使用特定的版本号替换curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt部分

curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.18.1/bin/linux/amd64/kubectl

2、执行权限以及移动到执行目录:

chmod +x ./kubectl
sudo cp ./kubectl /usr/local/bin/kubectl

3、查看版本号:

kubectl version --client

Docker安装与配置link

bbhsky阅读(641)

之前写过一篇CentOS下安装Docker的文章

突然在安装rancher2的时候,发现了一个笔记,做个笔记

1、Docker安装

Ubuntu 16.x

修改系统源

sudo cp /etc/apt/sources.list /etc/apt/sources.list.bak
cat > /etc/apt/sources.list << EOF

deb http://mirrors.aliyun.com/ubuntu/ xenial main
deb-src http://mirrors.aliyun.com/ubuntu/ xenial main
deb http://mirrors.aliyun.com/ubuntu/ xenial-updates main
deb-src http://mirrors.aliyun.com/ubuntu/ xenial-updates main
deb http://mirrors.aliyun.com/ubuntu/ xenial universe
deb-src http://mirrors.aliyun.com/ubuntu/ xenial universe
deb http://mirrors.aliyun.com/ubuntu/ xenial-updates universe
deb-src http://mirrors.aliyun.com/ubuntu/ xenial-updates universe
deb http://mirrors.aliyun.com/ubuntu/ xenial-security main
deb-src http://mirrors.aliyun.com/ubuntu/ xenial-security main
deb http://mirrors.aliyun.com/ubuntu/ xenial-security universe
deb-src http://mirrors.aliyun.com/ubuntu/ xenial-security universe

EOF

Docker-ce安装

# 定义用户名
NEW_USER=rancher
# 添加用户(可选)
sudo adduser $NEW_USER
# 为新用户设置密码
sudo passwd $NEW_USER
# 为新用户添加sudo权限
sudo echo "$NEW_USER ALL=(ALL) ALL" >> /etc/sudoers
# 定义安装版本
export docker_version=18.06.3;
# step 1: 安装必要的一些系统工具
sudo apt-get remove docker docker-engine docker.io containerd runc -y;
sudo apt-get update;
sudo apt-get -y install apt-transport-https ca-certificates \
    curl software-properties-common bash-completion  gnupg-agent;
# step 2: 安装GPG证书
sudo curl -fsSL http://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | \
    sudo apt-key add -;
# Step 3: 写入软件源信息
sudo add-apt-repository "deb [arch=amd64] http://mirrors.aliyun.com/docker-ce/linux/ubuntu \
    $(lsb_release -cs) stable";
# Step 4: 更新并安装 Docker-CE
sudo apt-get -y update;
version=$(apt-cache madison docker-ce|grep ${docker_version}|awk '{print $3}');
# --allow-downgrades 允许降级安装
sudo apt-get -y install docker-ce=${version} --allow-downgrades;
# 把当前用户加入docker组
sudo usermod -aG docker $NEW_USER;
# 设置开机启动
sudo systemctl enable docker;

Docker-engine

Docker-Engine Docker官方已经不推荐使用,请安装Docker-CE。

CentOS 7.x

Docker-ce安装

因为CentOS的安全限制,通过RKE安装K8S集群时候无法使用root账户。所以,建议CentOS用户使用非root用户来运行docker,不管是RKE还是custom安装k8s,详情查看无法为主机配置SSH隧道

# 定义用户名
NEW_USER=rancher
# 添加用户(可选)
sudo adduser $NEW_USER
# 为新用户设置密码
sudo passwd $NEW_USER
# 为新用户添加sudo权限
sudo echo "$NEW_USER ALL=(ALL) ALL" >> /etc/sudoers
# 卸载旧版本Docker软件
sudo yum remove docker \
              docker-client \
              docker-client-latest \
              docker-common \
              docker-latest \
              docker-latest-logrotate \
              docker-logrotate \
              docker-selinux \
              docker-engine-selinux \
              docker-engine \
              container*
# 定义安装版本
export docker_version=18.06.3
# step 1: 安装必要的一些系统工具
sudo yum update -y;
sudo yum install -y yum-utils device-mapper-persistent-data \
    lvm2 bash-completion;
# Step 2: 添加软件源信息
sudo yum-config-manager --add-repo \
    http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo;
# Step 3: 更新并安装 Docker-CE
sudo yum makecache all;
version=$(yum list docker-ce.x86_64 --showduplicates | sort -r|grep ${docker_version}|awk '{print $2}');
sudo yum -y install --setopt=obsoletes=0 docker-ce-${version} docker-ce-selinux-${version};
# 如果已经安装高版本Docker,可进行降级安装(可选)
yum downgrade --setopt=obsoletes=0 -y docker-ce-${version} docker-ce-selinux-${version};
# 把当前用户加入docker组
sudo usermod -aG docker $NEW_USER;
# 设置开机启动
sudo systemctl enable docker;

Docker-engine

Docker-Engine Docker官方已经不推荐使用,请安装Docker-CE。

[Rancher2]基础环境配置

bbhsky阅读(730)

1、系统配置要求

Rancher在以下操作系统及其后续的非主要发行版上受支持:

Ubuntu 16.04.x (64-bit)
Docker 18.06.x, 18.09.x
Ubuntu 18.04.x (64-bit)
Docker 18.06.x, 18.09.x
RancherOS 1.3.x+ (64-bit)
Docker 18.06.x, 18.09.x
Windows Server version 1803 (64-bit)
Docker 17.06
1、Ubuntu、Centos操作系统有Desktop和Server版本,选择请安装server版本,别自己坑自己! 
2、如果您正在使用RancherOS,请确保切换到受支持的Docker版本:
sudo ros engine switch docker-18.09.2

2、主机名配置

因为K8S的规定,主机名只支持包含 - 和 .(中横线和点)两种特殊符号,并且主机名不能出现重复。

3、Hosts

配置每台主机的hosts(/etc/hosts),添加host_ip $hostname/etc/hosts文件中。

4、CentOS关闭selinux

sudo sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

5、关闭防火墙(可选)或者放行相应端口

对于刚刚接触Rancher的用户,建议在关闭防火墙的测试环境或桌面虚拟机来运行rancher,以避免出现网络通信问题。

关闭防火墙

1、CentOS

systemctl stop firewalld.service && systemctl disable firewalld.service

2、Ubuntu

ufw disable

6、配置主机时间、时区、系统语言

  • 查看时区date -R或者timedatectl
  • 修改时区ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
  • 修改系统语言环境sudo echo 'LANG="en_US.UTF-8"' >> /etc/profile;source /etc/profile
  • 配置主机NTP时间同步

7、Kernel性能调优

cat >> /etc/sysctl.conf<<EOF
net.ipv4.ip_forward=1
net.bridge.bridge-nf-call-iptables=1
net.ipv4.neigh.default.gc_thresh1=4096
net.ipv4.neigh.default.gc_thresh2=6144
net.ipv4.neigh.default.gc_thresh3=8192
EOF

数值根据实际环境自行配置,最后执行sysctl -p保存配置。

8、内核模块

警告如果要使用ceph存储相关功能,需保证worker节点加载RBD模块

以下模块需要在主机上加载

模块名称
br_netfilter
ip6_udp_tunnel
ip_set
ip_set_hash_ip
ip_set_hash_net
iptable_filter
iptable_nat
iptable_mangle
iptable_raw
nf_conntrack_netlink
nf_conntrack
nf_conntrack_ipv4
nf_defrag_ipv4
nf_nat
nf_nat_ipv4
nf_nat_masquerade_ipv4
nfnetlink
udp_tunnel
VETH
VXLAN
x_tables
xt_addrtype
xt_conntrack
xt_comment
xt_mark
xt_multiport
xt_nat
xt_recent
xt_set
xt_statistic
xt_tcpudp

模块查询: lsmod | grep <模块名> 
模块加载: modprobe <模块名>

9、ETCD集群容错表

建议在ETCD集群中使用奇数个成员,通过添加额外成员可以获得更高的失败容错。具体详情可以查阅optimal-cluster-size

集群大小MAJORITY失败容错
110
220
321
431
532
642
743
853
954

设置HTTP/HTTPS 代理

bbhsky阅读(603)

1. 创建 docker.service.d 目录

mkdir -p /etc/systemd/system/docker.service.d

2. 创建HTTP 或者HTTPS 代理文件

# HTTP:
vim /etc/systemd/system/docker.service.d/http-proxy.conf

[Service]
Environment="HTTP_PROXY=xxx.xxx.xxx.xxx:443" "NO_PROXY=localhost,127.0.0.1,xxx.xxxxxx:5000"

# HTTPS:
vim /etc/systemd/system/docker.service.d/http-proxy.conf

[Service]
Environment="HTTPS_PROXY=xxx.xxx.xxx.xxx:443" "NO_PROXY=localhost,127.0.0.1,xxx.xxxxxx:5000"

3、完成修改后保存/刷新

systemctl daemon-reload
systemctl restart docker

4、查看修改结果

systemctl show --property=Environment docker

Environment=HTTPS_PROXY=xxx.xxx.xxx.xxx:443 NO_PROXY=localhost,127.0.0.1,mydocker-registry.com:5000

Jenkins报错

bbhsky阅读(1171)

今天遇见了Jenkins报错无法启动,查看了rancher没有任何报错只是一直重启,所以手动启动了一个镜像

docker run -d -p 8002:8080 -v /data/docker/data/jenkins/jenkins_home:/var/jenkins_home --name jenkins-bak --restart=always jenkins:2.46.2

发现以下报错

org.xmlpull.v1.XmlPullParserException: only 1.0 is supported as <?xml version not '1.1' (position: START_DOCUMENT seen <?xml version=\'1.1\'... @1:19) 
	at org.xmlpull.mxp1.MXParser.parseXmlDeclWithVersion(MXParser.java:2608)
	at org.xmlpull.mxp1.MXParser.parseXmlDecl(MXParser.java:2592)
	at org.xmlpull.mxp1.MXParser.parsePI(MXParser.java:2466)
	at org.xmlpull.mxp1.MXParser.parseProlog(MXParser.java:1447)
	at org.xmlpull.mxp1.MXParser.nextImpl(MXParser.java:1395)
	at org.xmlpull.mxp1.MXParser.next(MXParser.java:1093)
	at com.thoughtworks.xstream.io.xml.XppReader.pullNextEvent(XppReader.java:109)
Caused: com.thoughtworks.xstream.io.StreamException:  : only 1.0 is supported as <?xml version not '1.1' (position: START_DOCUMENT seen <?xml version=\'1.1\'... @1:19) 
	at com.thoughtworks.xstream.io.xml.XppReader.pullNextEvent(XppReader.java:124)
	at com.thoughtworks.xstream.io.xml.AbstractPullReader.readRealEvent(AbstractPullReader.java:148)
	at com.thoughtworks.xstream.io.xml.AbstractPullReader.readEvent(AbstractPullReader.java:141)
	at com.thoughtworks.xstream.io.xml.AbstractPullReader.move(AbstractPullReader.java:118)
	at com.thoughtworks.xstream.io.xml.AbstractPullReader.moveDown(AbstractPullReader.java:103)
	at com.thoughtworks.xstream.io.xml.XppReader.<init>(XppReader.java:63)
	at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:54)
	at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:65)
	at hudson.XmlFile.unmarshal(XmlFile.java:159)
Caused: java.io.IOException: Unable to read /var/jenkins_home/config.xml
	at hudson.XmlFile.unmarshal(XmlFile.java:161)
	at jenkins.model.Jenkins.loadConfig(Jenkins.java:3048)
	at jenkins.model.Jenkins.access$1200(Jenkins.java:307)
	at jenkins.model.Jenkins$16.run(Jenkins.java:3066)
	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
	at jenkins.model.Jenkins$7.runTask(Jenkins.java:1089)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused: org.jvnet.hudson.reactor.ReactorException
	at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:269)
	at jenkins.InitReactorRunner.run(InitReactorRunner.java:47)
	at jenkins.model.Jenkins.executeReactor(Jenkins.java:1113)
	at jenkins.model.Jenkins.<init>(Jenkins.java:929)
	at hudson.model.Hudson.<init>(Hudson.java:86)
	at hudson.model.Hudson.<init>(Hudson.java:82)
	at hudson.WebAppMain$3.run(WebAppMain.java:231)
Caused: hudson.util.HudsonFailedToLoad
	at hudson.WebAppMain$3.run(WebAppMain.java:248)

根据第一行,报错得出,是协议问题,修改了config.xml

<?xml version='1.1' encoding='UTF-8'?>
#改为
<?xml version='1.0' encoding='UTF-8'?>

然后重启Docker镜像

docker restart jenkins-bak

问题解决,还没搞清楚为什么文件头会从1.1协议变成了1.0~

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock:
#将jenkins用户加入docker组
#重启Jenkins服务
sudo gpasswd -a jenkins docker
sudo service jenkins restart

Rancher常见问题

bbhsky阅读(518)

1、ingress限制了上传大小:

#修改每个项目负载均衡YAML添加
nginx.ingress.kubernetes.io/proxy-body-size: 50m
#也可以点击编辑(Edit)→标签注释(Labels & Annotations)→注释(Annotations)
nginx.ingress.kubernetes.io/proxy-body-size = 50m

2、清理rancher

df -h|grep kubelet |awk -F % '{print $2}'|xargs umount 
rm /var/lib/kubelet/* -rf
rm /etc/kubernetes/* -rf
rm /var/lib/rancher/* -rf
rm /var/lib/etcd/* -rf
rm /var/lib/cni/* -rf
iptables -F && iptables -t nat –F
ip link del flannel.1
docker ps -a|awk '{print $1}'|xargs docker rm -f
docker volume ls|awk '{print $2}'|xargs docker volume rm

[Docker]安装Gogs

bbhsky阅读(762)

Docker安装

docker pull gogs/gogs
docker run -d --name=FoolTiger-Gogs -p 10022:22 -p 10080:3000 -v /data/docker/gogs:/data

打开http://192.168.50.1:10080

根据配置配置Gogs然后确定启动,这里切记,最好配置一下管理账号

如果想修改配置可以到/data/docker/gogs/conf/app.ini修改