ceph集群搭建

作者林文杰；结构图如下图所示

ceph.png

1、Selinux、防火墙和hostname文件

将/etc/selinux/config文件下SELINUX=disabled

关闭防火墙或者开通6789和6800：7300的端口

1
vim /etc/hostname

2、配置用户

1
2
3
4
5
6
useradd -m bdipceph
passwd bdipceph

#设置su权限
echo "bdipceph ALL = (root) NOPASSWD:ALL" | tee /etc/sudoers.d/bdipceph
chmod 0440 /etc/sudoers.d/bdipceph

3、将各节点写入hosts

1
2
3
echo 192.168.140.130 ceph130 >> /etc/hosts
echo 192.168.140.133 ceph133 >> /etc/hosts
echo 192.168.140.134 ceph134 >> /etc/hosts

4、配置免密

1
2
3
4
5
6

su bdipceph
ssh-keygen -t rsa
ssh-copy-id ceph130
ssh-copy-id ceph133
ssh-copy-id ceph134

5、安装ntp服务和配置yum仓库

1
2
vim /etc/yum.repo.d/CentOS-ceph.repo

下面做参考

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
[Ceph]
name=Ceph packages for $basearch
baseurl=https://mirrors.aliyun.com/ceph/rpm-jewel/el7/$basearch
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc
priority=1
[Ceph-noarch]
name=Ceph noarch packages
baseurl=https://mirrors.aliyun.com/ceph/rpm-jewel/el7/noarch
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc
priority=1
[ceph-source]
name=Ceph source packages
baseurl=https://mirrors.aliyun.com/ceph/rpm-jewel/el7/SRPMS
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc

6、安装存储集群

1
2
3
4
5
6
7
8
9
10
11
12
13

yum install ceph-deploy
su cephd
mkdir ~/my-cluster
ceph-deploy new ceph133
osd pool default size = 2
ceph-deploy install ceph130 ceph133 ceph134
ceph-deploy mon create-initial
ceph-deploy osd prepare ceph133:/var/local/osd0 ceph134:/var/local/osd1
ceph-deploy osd activate ceph133:/var/local/osd0 ceph134:/var/local/osd1
ceph-deploy admin ceph130 ceph133 ceph134
chmod +r /etc/ceph/ceph.client.admin.keyring
ceph health

7、删除osd

1
2
3
4
5
6
7
8
9
ceph osd out 1
systemctl stop ceph-osd@1
ceph osd crush remove osd.1
ceph auth del osd.1
ceph osd rm osd.1
#删除旧文件夹
rm -rf /var/local/osd1
#调整权重
ceph osd crush reweight osd.1 0

8、配置文件修改发送

1
ceph-deploy --overwrite-conf config push bdipceph104 bdipceph106 bdipceph108

9、配置文件内容

1
2
3
4
5
6
7
8
9
#设置日志文件大小
osd journal size = 1024
#设置节点数量
osd pool default size = 3
#设置副本数量
osd pool default min size = 2
#设置ext4能够使用
osd max object name len = 256
osd max object namespace len = 64

10、创建块设备

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
#创建映像foo大小4G
rbd create --size 4096 foo
#查询映像
rbd ls
#查询单个映像信息
rbd info foo
#扩大映像大小
rbd resize --size 8192 foo
resize2fs /dev/rbd0
#减小映像大小(会损坏数据)
rbd resize --size 4096 foo --allow-shrink
#删除映像
rbd rm foo
#linux环境需要执行下面这条命令
rbd feature disable foo exclusive-lock object-map fast-diff deep-flatten
#映射块设备
rbd map foo --name client.admin 
#查看已映射块设备
rbd showmapped

#取消块设备映射
rbd unmap /dev/rbd0
mkfs.ext4 /dev/rbd0

11、创建块设备快照

1
2
3
4
5
6
7
8
9
10
#创建快照
rbd snap create rbd/foo@snapfoo
#罗列快照
rbd snap ls rbd/foo
#回滚快照
rbd snap rollback rbd/foo@snapfoo
#删除快照
rbd snap rm rbd/foo@snapfoo
#清除快照
rbd snap purge rbd/foo
大数据
#ceph
ceph集群搭建
http://example.com/2019/07/17/2019-07-17-ceph/
Author
张帅
Posted on
July 17, 2019
Licensed under