简介

1.简介

SmallComputer System Interface(SCSI,小型计算机系统接口),是一种用于计算机及其外围设备之间(硬盘、光驱、打印机、扫描仪)的物理的连接和传输数据的一套标准。SCSI标准定义命令、通讯协以及实体的电气特性,最长应用于存储设备上。是一种广泛应用于小型机上的高速数据传输技术。SCSI接口具有应用范围广、多任务、带宽大、CPU占用率低,以及热插拔等优点。SCSI硬盘主要应用于中、高端服务器和高档工作站中。在系统中应用SCSI必须要有专门的SCSI控制器,也就是一块SCSI控制卡,才能支持SCSI设备,SCSI控制器有自己的命令集和缓存。SCSI控制器上有一个相当于CPU的芯片,它对SCSI设备进行控制,能处理大部分的工作,减少了中央处理器的负担(CPU占用率)。

2.工作流程

SCSI层:根据客户端发出的请求建立SCSICDB(命令描述块),并传给ISCSI层。同时接收来着ISCSI层的CDB,并向应用层返回数据。
ISCSI层:对SCSI CDB进行封装,以便能够在基于TCP/IP协议的网络上进行传输,完成SCSI到TCP/IP的协议转换。
TCP/IP层:对IP报文进行路由和转发。
ISCSI协议定义了在TCP/IP网络发送、接收数据块存储数据的规则和方式。先发送端将SCSI命令和数据封装到TCP/IP包中,然后通过IP网络转发,接收端收到TCP/IP包后,将其还原为SCSI命令和数据并执行,执行完后,将返回的SCSI命令和数据再封装到TCP/IP包中,之后再传回发送端。

wKiom1VlrePhDgfcAACz2T-JSz0586.jpg

实验

1.创建一个最简单的共享磁盘

服务端 (192.168.10.13)

0.准备一个磁盘(可选)
yum install -y mdadm
mdadm -C /dev/md10 -l 10 -n 3 /dev/sdb{1..3}
1.安装 iSCSI 相关服务
yum install targetcli -y
2.创建基本共享磁盘
targetcli
    backstores/block create test1 /dev/md10        #创建目标存储 并重命名test1,后面跟的资源需要是本服务器拥有的存储块
    iscsi/ create iqn.2020-11.cn.test:server    #创建iscsi 格式iqn.年份-月份.根域.一级域名:名称<不能有下划线>
    iscsi/iqn.2020-11.cn.test:server/tpg1/luns create /backstores/block/test1    ## lun配置,链接磁盘
    iscsi/iqn.2020-11.cn.test:server/tpg1/acls create iqn.2020-11.cn.test:server    ## acl配置,一定要创建,内容可以不写
    cd /
    saveconfig    #保存配置
    exit
3.查看
/> ls
o- / .................................................................................................................. [...]
  o- backstores ....................................................................................................... [...]
  | o- block ........................................................................................... [Storage Objects: 1]
  | | o- test1 .................................................................... [/dev/md10 (3.0GiB) write-thru activated]
  | |   o- alua ............................................................................................ [ALUA Groups: 1]
  | |     o- default_tg_pt_gp ................................................................ [ALUA state: Active/optimized]
  | o- fileio .......................................................................................... [Storage Objects: 0]
  | o- pscsi ........................................................................................... [Storage Objects: 0]
  | o- ramdisk ......................................................................................... [Storage Objects: 0]
  o- iscsi ..................................................................................................... [Targets: 1]
  | o- iqn.2020-11.cn.test:server ................................................................................. [TPGs: 1]
  |   o- tpg1 ........................................................................................ [no-gen-acls, no-auth]
  |     o- acls ................................................................................................... [ACLs: 0]
  |     o- luns ................................................................................................... [LUNs: 1]
  |     | o- lun0 .............................................................. [block/test1 (/dev/md10) (default_tg_pt_gp)]
  |     o- portals ............................................................................................. [Portals: 1]
  |       o- 0.0.0.0:3260 .............................................................................................. [OK]
  o- loopback .................................................................................................. [Targets: 0]

image-20201117150628634

tpg1
    -luns    # 链接到哪个资源
    -acls    # 链接规则
    -portals    # 访问地址范围

客户端 (192.168.10.14)

1.安装iSCSI链接程序
yum install -y iscsi-init*
2.发现 & 链接 iSCSi
## 1.先发现
iscsiadm -m discovery -t st -p 192.168.10.13 
192.168.10.13:3260,1 iqn.2020-11.cn.test:server        # 输出信息,找到了服务端的目标存储

## 2.修改配置文件
vim /etc/iscsi/initiatorname.iscsi
    InitiatorName=iqn.2020-11.cn.test:server        # InitiatorName=iqn名称
    
## 3.后登陆
iscsiadm -m discovery -t st -p 192.168.10.13 -l        #发现192.168.10.13下的全部目标存储
iscsiadm --m node –login    #登录发现的全部目标存储

## 4.查看(测试用)
## 法1
[root@localhost ~]# lsblk
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0   20G  0 disk 
├─sda1            8:1    0    1G  0 part /boot
└─sda2            8:2    0   19G  0 part 
  ├─centos-root 253:0    0   17G  0 lvm  /
  └─centos-swap 253:1    0    2G  0 lvm  [SWAP]
sdb               8:16   0    3G  0 disk     # 此盘就是 
sr0              11:0    1  4.2G  0 rom  /media

## 法2
[root@localhost ~]# lsscsi -ds               
[0:0:0:0]    disk    VMware,  VMware Virtual S 1.0   /dev/sda [8:0]  21.4GB
[2:0:0:0]    cd/dvd  NECVMWar VMware IDE CDR10 1.00  /dev/sr0 [11:0]       -
[14:0:0:0]   disk    LIO-ORG  test1            4.0   /dev/sdb [8:16]  3.21GB

## 法3
[root@localhost ~]# iscsiadm -m session -P 3    # -P 查看第三个磁盘
iSCSI Transport Class version 2.0-870
version 6.2.0.874-2
Target: iqn.2020-11.cn.test:server (non-flash)
        Current Portal: 192.168.10.13:3260,1
        Persistent Portal: 192.168.10.13:3260,1
                **********
                Interface:
                **********
                Iface Name: default
                Iface Transport: tcp
                Iface Initiatorname: iqn.2020-11.cn.test:server
                Iface IPaddress: 192.168.10.14
                Iface HWaddress: <empty>
                Iface Netdev: <empty>
                SID: 12
                iSCSI Connection State: LOGGED IN
                iSCSI Session State: LOGGED_IN
                Internal iscsid Session State: NO CHANGE
                *********
                Timeouts:
                *********
                Recovery Timeout: 120
                Target Reset Timeout: 30
                LUN Reset Timeout: 30
                Abort Timeout: 15
                *****
                CHAP:
                *****
                username: <empty>
                password: ********
                username_in: <empty>
                password_in: ********
                ************************
                Negotiated iSCSI params:
                ************************
                HeaderDigest: None
                DataDigest: None
                MaxRecvDataSegmentLength: 262144
                MaxXmitDataSegmentLength: 262144
                FirstBurstLength: 65536
                MaxBurstLength: 262144
                ImmediateData: Yes
                InitialR2T: Yes
                MaxOutstandingR2T: 1
                ************************
                Attached SCSI devices:
                ************************
                Host Number: 14 State: running
                scsi14 Channel 00 Id 0 Lun: 0
                        Attached scsi disk sdb          State: running
3.格式化 & 使用 & 挂载
mkfs.xfs /dev/sdb
mkdir /iscsi
mount /dev/sdb /iscsi/
df -h
[root@localhost ~]# df -h
文件系统                 容量  已用  可用 已用% 挂载点
/dev/mapper/centos-root   17G 1022M   16G    6% /
devtmpfs                 478M     0  478M    0% /dev
tmpfs                    489M     0  489M    0% /dev/shm
tmpfs                    489M  6.8M  482M    2% /run
tmpfs                    489M     0  489M    0% /sys/fs/cgroup
/dev/sr0                 4.3G  4.3G     0  100% /media
/dev/sda1               1014M  125M  890M   13% /boot
tmpfs                     98M     0   98M    0% /run/user/0
/dev/sdb                 3.0G   33M  3.0G    2% /iscsi        # 挂载成功
4.断开 & 登出
umount /iscsi
[root@localhost ~]# iscsiadm -m node -u        # 登出全部目标存储
Logging out of session [sid: 12, target: iqn.2020-11.cn.test:server, portal: 192.168.10.13,3260]
Logout of [sid: 12, target: iqn.2020-11.cn.test:server, portal: 192.168.10.13,3260] successful.
[root@localhost ~]# lsblk
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0   20G  0 disk 
├─sda1            8:1    0    1G  0 part /boot
└─sda2            8:2    0   19G  0 part 
  ├─centos-root 253:0    0   17G  0 lvm  /
  └─centos-swap 253:1    0    2G  0 lvm  [SWAP]
sr0              11:0    1  4.2G  0 rom  /media

2.iSCSI 双向Chap验证

下面的操作连贯着上面的配置

服务端

1.Session双向Chap验证
targetcli
    cd /iscsi/iqn.2020-11.cn.test:server/tpg1/acls/
    create iqn.2020-11.cn.test:server    # 名字跟iqn一样
    cd iqn.2020-11.cn.test:server/
    set auth userid=session password=sessionpassword mutual_userid=session_in mutual_password=sessioninpass
    # 下面是输出信息
    Parameter userid is now 'session'.
    Parameter password is now 'sessionpassword'.
    Parameter mutual_password is now 'sessioninpass'.
    Parameter mutual_userid is now 'session_in'.

Session Chap:iSCSi客户端请求发现完服务端,进行登录时为Session

  • userid,password 是服务端验证客户端的账号密码,即服务端验证客户端是否合法
  • mutual_userid,mutual_password 是客户端验证服务端的账号密码,即客户端验证务端是否合法
2.Discovery双向Chap认证
targetcli
    cd /iscsi/
    set discovery_auth enable=1 userid=discovery password=discoverypass mutual_userid=discovery_in mutual_password=discoveryinpass
    # 下面是输出信息
    Parameter userid is now 'discovery'.
    Parameter password is now 'discoverypass'.
    Parameter mutual_password is now 'discoveryinpass'.
    Parameter mutual_userid is now 'discovery_in'.
    Parameter enable is now 'True'.

Discovery Chap:iSCSi客户端请求发现服务端中的目标存储,去查找服务端有哪些目标存储为Discovery

  • userid,password 是服务端验证客户端的账号密码,即服务端验证客户端是否合法
  • mutual_userid,mutual_password 是客户端验证服务端的账号密码,即客户端验证务端是否合法
3.设定监控端口
targetcli
    cd /iscsi/iqn.2020-11.cn.test:server/tpg1/portals/
    delete 0.0.0.0 3260
    create 192.168.10.13 3260    # IP + 端口 ,用以实现通信时专门使用iSCSI网卡
4.查看
 ls /
o- / .................................................................................................................. [...]
  o- backstores ....................................................................................................... [...]
  | o- block ........................................................................................... [Storage Objects: 1]
  | | o- test1 .................................................................... [/dev/md10 (3.0GiB) write-thru activated]
  | |   o- alua ............................................................................................ [ALUA Groups: 1]
  | |     o- default_tg_pt_gp ................................................................ [ALUA state: Active/optimized]
  | o- fileio .......................................................................................... [Storage Objects: 0]
  | o- pscsi ........................................................................................... [Storage Objects: 0]
  | o- ramdisk ......................................................................................... [Storage Objects: 0]
  o- iscsi ................................................................................... [mutual disc auth, Targets: 1]
  | o- iqn.2020-11.cn.test:server ................................................................................. [TPGs: 1]
  |   o- tpg1 ........................................................................................ [no-gen-acls, no-auth]
  |     o- acls ................................................................................................... [ACLs: 1]
  |     | o- iqn.2020-11.cn.test:server .................................................................... [Mapped LUNs: 1]
  |     |   o- mapped_lun0 .......................................................................... [lun0 block/test1 (rw)]
  |     o- luns ................................................................................................... [LUNs: 1]
  |     | o- lun0 .............................................................. [block/test1 (/dev/md10) (default_tg_pt_gp)]
  |     o- portals ............................................................................................. [Portals: 1]
  |       o- 192.168.10.14:3260 ......................................................................................... [OK]
  o- loopback .................................................................................................. [Targets: 0]

image-20201117161108788

客户端

1.修改iSCSI默认配置文件
yum install -y iscsi*
vim /etc/iscsi/iscsid.conf 
/auth     # 找到auth等关键字
    ## Session Chap 验证
    node.session.auth.authmethod = CHAP
    node.session.auth.username = session
    node.session.auth.password = sessionpassword
    node.session.auth.username_in = session_in
    node.session.auth.password_in = sessioninpass
    ## Session Chap 验证
    discovery.sendtargets.auth.authmethod = CHAP
    discovery.sendtargets.auth.username = discovery
    discovery.sendtargets.auth.password = discoverypass
    discovery.sendtargets.auth.username_in = discovery_in
    discovery.sendtargets.auth.password_in = discoveryinpass

演示

# *************
# CHAP Settings
# *************

# To enable CHAP authentication set node.session.auth.authmethod
# to CHAP. The default is None.
node.session.auth.authmethod = CHAP

# To set a CHAP username and password for initiator
# authentication by the target(s), uncomment the following lines:
node.session.auth.username_in = session_in
node.session.auth.password_in = sessioninpass

# To set a CHAP username and password for target(s)
# authentication by the initiator, uncomment the following lines:
node.session.auth.username_in = session_in
node.session.auth.password_in = sessioninpass

# To enable CHAP authentication for a discovery session to the target
# set discovery.sendtargets.auth.authmethod to CHAP. The default is None.
discovery.sendtargets.auth.authmethod = CHAP

# To set a discovery session CHAP username and password for the initiator
# authentication by the target(s), uncomment the following lines:
discovery.sendtargets.auth.username = discovery
discovery.sendtargets.auth.password = discoverypass

# To set a discovery session CHAP username and password for target(s)
# authentication by the initiator, uncomment the following lines:
discovery.sendtargets.auth.username_in = discovery_in
discovery.sendtargets.auth.password_in = discoveryinpass
2.链接测试
## 发现 & 登陆
systemctl restart iscsi*
iscsiadm -m discovery -t st -p 192.168.10.13 -l
iscsiadm --m node –login

[root@localhost ~]# iscsiadm -m session -P 3
iSCSI Transport Class version 2.0-870
version 6.2.0.874-2
Target: iqn.2020-11.cn.test:server (non-flash)
        Current Portal: 192.168.10.13:3260,1
        Persistent Portal: 192.168.10.13:3260,1
                **********
                Interface:
                **********
                Iface Name: default
                Iface Transport: tcp
                Iface Initiatorname: iqn.2020-11.cn.test:server
                Iface IPaddress: 192.168.10.14
                Iface HWaddress: <empty>
                Iface Netdev: <empty>
                SID: 13
                iSCSI Connection State: LOGGED IN
                iSCSI Session State: LOGGED_IN
                Internal iscsid Session State: NO CHANGE
                *********
                Timeouts:
                *********
                Recovery Timeout: 120
                Target Reset Timeout: 30
                LUN Reset Timeout: 30
                Abort Timeout: 15
                *****
                CHAP:            ## 这里下面的即为登陆账号
                *****
                username: session
                password: ********
                username_in: session_in
                password_in: ********
                ************************
                Negotiated iSCSI params:
                ************************
                HeaderDigest: None
                DataDigest: None
                MaxRecvDataSegmentLength: 262144
                MaxXmitDataSegmentLength: 262144
                FirstBurstLength: 65536
                MaxBurstLength: 262144
                ImmediateData: Yes
                InitialR2T: Yes
                MaxOutstandingR2T: 1
                ************************
                Attached SCSI devices:
                ************************
                Host Number: 15 State: running
                scsi15 Channel 00 Id 0 Lun: 0
                    Attached scsi disk sdc          State: running

3.多路径

1.安装多路径服务
yum install -y device-mapper*
2.生成多路径配置文件
systemctl restart iscsi* multipath*
## 在CentOS7中启用多路径模块(生成配置文件、加载DM模块)
mpathconf --enable
3.修改默认文件,使其显示目标存储
vim /etc/multipath.conf
    defaults {
            user_friendly_names yes
            find_multipaths no    # 将此处修改为no
            ## 因为multipath -l 查询时只查询/etc/multipath/wwids文件内有的wwid目标存储。
    }
    
[root@localhost ~]# multipath -r    # 重载配置文件
[root@localhost ~]# multipath -l    # 查看并记录iscsi设备的wwid号
mpatha (3600140541c8dd8e65844235b3280a9bd) dm-2 LIO-ORG ,test1           
size=1.5G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=0 status=active
  `- 16:0:0:0 sdb 8:16 active undef unknown
4.编辑配置文件
vim /etc/multipath.conf
    multipaths {
        multipath {
            wwid                    3600140541c8dd8e65844235b3280a9bd    #这是刚才记录的wwid号
            alias                   abab    #配置显示别名
            path_selector           "round-robin 0"    #轮巡策略
                   }
               }
               
[root@localhost ~]# multipath -r
Nov 18 20:55:08 | 3600140541c8dd8e65844235b3280a9bd: rename mpatha to abab
: abab (3600140541c8dd8e65844235b3280a9bd) undef LIO-ORG ,test1           
size=1.5G features='0' hwhandler='0' wp=undef
`-+- policy='round-robin 0' prio=1 status=undef
  `- 16:0:0:0 sdb 8:16 active ready running
## 信息已经更新
5.挂载使用
[root@localhost ~]# lsblk
sdb               8:16   0  1.5G  0 disk  
└─abab          253:2    0  1.5G  0 mpath     # 多路径,自定义别名
[root@localhost ~]# fdisk -l
磁盘 /dev/mapper/abab:1607 MB, 1607467008 字节,3139584 个扇区
Units = 扇区 of 1 * 512 = 512 bytes
扇区大小(逻辑/物理):512 字节 / 512 字节
I/O 大小(最小/最佳):524288 字节 / 1572864 字节
[root@localhost ~]# mkfs.xfs /dev/mapper/abab

配置文件扩展

defaults    #DM Multipath 的常规默认设置。
blacklist   #不被视为多路径的具体设备列表。
blacklist_exceptions    #根据 blacklist 部分中的参数列出不在黑名单中的多路径设备
multipaths  #各个独立多路径设备的特性设置。这些数值覆盖了在配置文件的 defaults 和 devices 部分中指定的数值
devices     #各个存储控制器的设置。这些数值覆盖了在配置文件的 defaults 部分指定的数值。如果要使用不是默认支持的存储阵列,则可能需要为您的阵列创建 devices 子部分

centos7 多路径设备支持官方文档

Last modification:June 28th, 2021 at 03:54 pm