CMDB项目

一 CMDB简介

https://www.wangt.cc/2021/03/cmdb%E9%A1%B9%E7%9B%AE/

1.1 什么是CMDB？

CMDB（资产管理系统）是所有运维工具的数据基础

1.2 CMDB包含的功能

用户管理，记录测试，开发，运维人员的用户表

业务线管理，需要记录业务的详情

项目管理，指定此项目用属于哪条业务线，以及项目详情

应用管理，指定此应用的开发人员，属于哪个项目，和代码地址，部署目录，部署集群，依赖的应用，软件等信息

主机管理，包括云主机，物理机，主机属于哪个集群，运行着哪些软件，主机管理员，连接哪些网络设备，云主机的资源池，存储等相关信息

主机变更管理，主机的一些信息变更，例如管理员，所属集群等信息更改，连接的网络变更等

网络设备管理，主要记录网络设备的详细信息，及网络设备连接的上级设备

IP管理，IP属于哪个主机，哪个网段, 是否被占用等

1.3 实现的四种方式

1.3.1 Agent实现方式

Agent方式，可以将服务器上面的Agent程序作定时任务，定时将资产信息提交到指定API录入数据库

其本质上就是在各个服务器上执行subprocess.getoutput()命令，然后将每台机器上执行的结果，返回给主机API，然后主机API收到这些数据之后，放入到数据库中，最终通过web界面展现给用户

1. #linux
2. import subprocess
3. import re
4. res = subprocess.getoutput("ifconfig")
5. print(res)
6. ip=re.findall('inet (.*?) netmask',res)
7. print(ip)
9. # windows
10. import subprocess
11. import re
12. res=subprocess.getoutput('ipconfig')
14. print(res)
15. ip=re.findall('IPv4 地址 . . . . . . . . . . . . : (.*)',res)
16. print(ip)

优点：速度快
缺点：需要为每台服务器部署一个Agent程序

使用crontab定时执行python脚本

1. # 1 进入创建crontab定时任务
2. crontab -e
3. # 2 写入任务（每分钟执行一次test.py）
4. * * * * * python3 test.py
5. # 3 编写test.py
6. with open('a.txt','a') as f:
7. f.write('hello world')
9. # 4 查看定时任务
10. crontab -l

---

1.3.2 ssh实现方式 （基于Paramiko模块）

中控机通过Paramiko（py模块）登录到各个服务器上，然后执行命令的方式去获取各个服务器上的信息

优点：无Agent

缺点：速度慢

如果在服务器较少的情况下，可应用此方法

1. import paramiko
2. import re
3. #创建SSH对象
4. ssh = paramiko.SSHClient()
6. # 允许连接不在know_hosts文件中的主机
7. ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy)
9. # 连接服务器
10. ssh.connect(hostname='101.133.225.166',port=22,username='root',password='')
11. # 执行命令
12. stdin,stdout,stderr = ssh.exec_command('ifconfig')
14. # 获取命令结果
15. result = stdout.read().decode('utf-8')
17. print(result)
18. ip=re.findall('inet (.*?) netmask',result)
19. print(ip)
20. # 关闭连接
21. ssh.close()

1.3.3 saltstack方式

此方案本质上和第二种方案大致是差不多的流程，中控机发送命令给服务器执行。服务器将结果放入另一个队列中，中控机获取将服务信息发送到API进而录入数据库。

执行流程:
第一步: 由管理员录入资产(主机名,SN等信息),通过后台管理,录入数据库
第二步: salt-master从数据库获取未采集资产信息的服务器
第三步: salt-master发送命令给salt-minion执行
第四步: salt-master拿到执行结果
第五步: 将结果发送给API
第六步: API将其写入数据库

解释:
salt-master可以理解为主人
salt-minion可以理解为奴隶

优点：快，开发成本低

缺点：依赖于第三方工具

salstack的安装和配置

1.安装和配置

1. master端：
2. """
3. 1. 安装salt-master
4. yum install salt-master
5. 2. 修改配置文件：/etc/salt/master
6. interface: 0.0.0.0 # 表示Master的IP
7. 3. 启动
8. service salt-master start
9. """
10. slave端：
11. """
12. 1. 安装salt-minion
13. yum install salt-minion
14. 2. 修改配置文件 /etc/salt/minion
15. master: 10.211.55.4 # master的地址
16. 或
17. master:
18. - 10.211.55.4
19. - 10.211.55.5
20. random_master: True
21. id: c2.salt.com # 客户端在salt-master中显示的唯一ID
22. 3. 启动
23. service salt-minion start

2.授权

1. salt-key -L # 查看已授权和未授权的slave
2. salt-key -a salve_id # 接受指定id的salve
3. salt-key -r salve_id # 拒绝指定id的salve
4. salt-key -d salve_id # 删除指定id的salve

3.执行命令

在master服务器上对salve进行远程操作

1. salt 'c2.salt.com' cmd.run 'ifconfig'
2. # 基于API的方式

1. import salt.client
2. local = salt.client.LocalClient()
3. result = local.cmd('c2.salt.com', 'cmd.run', ['ifconfig'])

1.3.4 Puppet（ruby语言开发）（了解）

每隔30分钟，通过RPC消息队列将执行的结果返回给用户

二 三种方案客户端编写

2.1 目录结构划分

1. autoclient # 项目名
2. -bin # 启动文件路径
3. -start.py # 启动文件
4. -config # 配置文件路径
5. -cert # 私钥
6. -custom_settings.py # 用户自定义配置
7. -files # 测试数据文件
8. -board.out
9. -cpuinfo.out
10. -disk.out
11. -memory.out
12. -nic.out
13. -lib # 库文件夹
14. -conf # 配置信息文件夹
15. -config.py # 配置类
16. -global_settings.py # 全局常量配置
17. -convert.py # 公共方法
18. -src # 源文件
19. -plugins # 插件
20. -__init__.py # 初始化文件
21. -basic.py
22. -board.py
23. -cpu.py
24. -disk.py
25. -memory.py
26. -nic.py
27. script.py # 脚本文件
28. client.py # 客户端类
29. tests # 测试文件夹
32. # 总结：bin，config，files，lib，src几个文件夹

2.2 仿django配置文件

custom_settings.py

1. # 用户配置
2. PORT = 22
3. USER = 'lqz'

global_settings.py

1. #### 全局配置
3. PORT = 22
4. USER = 'root'

config.py

1. from config import custom_settings
2. from . import global_settings
4. class Settings():
5. def __init__(self):
7. #### 全局配置
8. for key in dir(global_settings):
9. if key.isupper():
10. #### 获取key所对应的值
11. v = getattr(global_settings, key)
12. #### 设置key以及值到当前的setting对象
13. setattr(self, key, v)
15. #### 自定制配置
16. for key in dir(custom_settings):
17. if key.isupper():
18. #### 获取key所对应的值
19. v = getattr(custom_settings, key)
20. #### 设置key以及值到当前的setting对象
21. setattr(self, key, v)
23. settings = Settings()

2.3 可插拔式配置

custom_settings.py

1. ### 可插拔式的采集,注释掉某个就不会执行
2. PLUGINS_DICT = {
3. 'basic':'src.plugins.basic.Basic',
4. 'board':'src.plugins.board.Board',
5. 'cpu':'src.plugins.cpu.Cpu',
6. 'disk':'src.plugins.disk.Disk',
7. 'nic':'src.plugins.nic.Nic',
8. 'memory':'src.plugins.memory.Memory',
9. }

src/plugins/__init__.py

1. import traceback
3. from lib.conf.config import settings
4. import importlib
5. import subprocess
6. ### 管理插件信息的类
7. class PluginsManager(object):
9. def __init__(self, hostname=None):
10. pass
12. ### 读取配置文件中的pluginsdict， 并执行对应模块中的process方法
13. def execute(self):
14. response = {}
15. for k, v in self.plugins_dict.items():
16. ret = {"status":None, 'data':None}
17. '''
18. k: board,...
19. v: src.plugins.board.Board 字符串
20. '''
21. try:
22. # 1. 导入模块路径
23. moudle_path, class_name = v.rsplit('.', 1)
24. # 2. 导入这个路径
25. moudle_name = importlib.import_module(moudle_path)
26. # 3. 导入对应模块下的类
27. classobj = getattr(moudle_name, class_name)
28. # 4. 执行类下面对应的process方法
29. res = classobj().process()
30. except Exception as e:
31. pass
32. return response

src/plugins/cpu.py

1. #!/usr/bin/env python
2. # -*- coding:utf-8 -*-
3. import os
4. from lib.conf.config import settings
6. class Cpu(object):
7. def __init__(self):
8. pass
10. @classmethod
11. def initial(cls):
12. return cls()
14. def process(self, command_func, debug):
15. print('cpu print')

src/plugins/disk.py

1. import os
2. from lib.conf.config import settings
4. class Cpu(object):
5. def __init__(self):
6. pass
8. @classmethod
9. def initial(cls):
10. return cls()
12. def process(self, command_func, debug):
13. print('disk print')

2.4 冗余代码抽取

继承方式

把函数当参数传入函数中:

在src/plugins/init.py中写，__隐藏，调用execute的时候，把函数地址和命令传入

1. import traceback
3. from lib.conf.config import settings
4. import importlib
5. import subprocess
6. ### 管理插件信息的类
7. class PluginsManager(object):
9. def __init__(self, hostname=None):
10. self.plugins_dict = settings.PLUGINS_DICT
11. self.hostname = hostname # 采集客户端的地址
12. self.debug = settings.DEBUG
13. if settings.MODE == 'ssh': # ssh方式才需要端口，用户名，密码，这些应该放到配置文件中
14. self.port = settings.SSH_PORT
15. self.name = settings.SSH_USERNAME
16. self.pwd = settings.SSH_PASSWORD
18. ### 读取配置文件中的pluginsdict， 并执行对应模块中的process方法
19. def execute(self):
20. response = {}
21. for k, v in self.plugins_dict.items():
22. ret = {"status":None, 'data':None}
23. '''
24. k: board,...
25. v: src.plugins.board.Board 字符串
26. '''
27. try:
28. # 1. 导入模块路径
29. moudle_path, class_name = v.rsplit('.', 1)
30. # 2. 导入这个路径
31. moudle_name = importlib.import_module(moudle_path)
32. # 3. 导入对应模块下的类
33. classobj = getattr(moudle_name, class_name)
34. # 4. 执行类下面对应的process方法
35. res = classobj().process(self.__cmd_run, self.debug)
36. ret['status'] = 10000
37. ret['data'] = res
38. except Exception as e:
39. ret['status'] = 10001
40. ret['data']= "[%s] 采集 [%s] 出错了， 错误信息是：%s" % (self.hostname if self.hostname else "Agent", k, str(traceback.format_exc()))
41. response[k] = ret
42. return response
45. def __cmd_run(self, cmd):
46. if settings.MODE == 'agent':
47. return self.__cmd_agent(cmd)
48. elif settings.MODE == 'ssh':
49. return self.__cmd_ssh(cmd)
50. elif settings.MODE == 'salt':
51. return self.__cmd_salt(cmd)
52. else:
53. print("只支持的模式有：agent/ssh/salt")
55. def __cmd_agent(self, cmd):
56. res = subprocess.getoutput(cmd)
57. return res
59. def __cmd_ssh(self, cmd):
60. import paramiko
61. # 创建SSH对象
62. ssh = paramiko.SSHClient()
63. # 允许连接不在know_hosts文件中的主机
64. ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
65. # 连接服务器
66. ssh.connect(hostname=self.hostname, port=self.port, username=self.name, password=self.pwd)
67. # 执行命令
68. stdin, stdout, stderr = ssh.exec_command(cmd)
69. # 获取命令结果
70. result = stdout.read()
71. # 关闭连接
72. ssh.close()
73. return result
75. def __cmd_salt(self, cmd):
76. command = "salt %s cmd.run %s" % (self.hostname, cmd)
77. res = subprocess.getoutput(command)
78. return res

在cpu.py disk.py中编写

1. class Cpu(object):
2. def __init__(self):
3. pass
5. @classmethod
6. def initial(cls):
7. return cls()
9. def process(self, command_func, debug):
10. if debug:
11. output = open(os.path.join(settings.BASEDIR, 'files/cpuinfo.out'), 'r', encoding='utf-8').read()
12. else:
13. output = command_func("cat /proc/cpuinfo")
14. return self.parse(output)

2.5 解析数据（以主板为例）

1. # sudo dmidecode -t1 https://ipcmen.com/dmidecode
2. # 可以获取BIOS，系统，主板，处理器，内存，缓存等 序列号、电脑厂商、串口信息以及其它系统配件信息
3. res = '''
4. SMBIOS 2.7 present.
6. Handle 0x0001, DMI type 1, 27 bytes
7. System Information
8. Manufacturer: Parallels Software International Inc.
9. Product Name: Parallels Virtual Platform
10. Version: None
11. Serial Number: Parallels-1A 1B CB 3B 64 66 4B 13 86 B0 86 FF 7E 2B 20 30
12. UUID: 3BCB1B1A-6664-134B-86B0-86FF7E2B2030
13. Wake-up Type: Power Switch
14. SKU Number: Undefined
15. Family: Parallels VM
16. '''
18. key_map = {
19. "Manufacturer" : 'manufacturer',
20. "Product Name" : 'product_name',
21. "Serial Number": 'sn'
22. }
24. result = {}
25. data = res.strip().split('n')
26. # print(data)
27. for k in data:
28. v = (k.strip().split(':'))
29. if len(v) == 2:
30. if v[0] in key_map:
31. result[key_map[v[0]]] = v[1].strip()
33. print(result)
35. '''
36. result = {
37. 'manufacturer' : 'Parallels Software International Inc.' ,
38. 'product_name' : 'Parallels Virtual Platform',
39. 'sn' : 'Parallels-1A 1B CB 3B 64 66 4B 13 86 B0 86 FF 7E 2B 20 30'
40. }
41. '''

2.6 代码整合

1. plugins
2. -__init__.py
3. -basic.py
4. -board.py
5. -cpu.py
6. -disk.py
7. -memory.py
8. -nic.py

1. #__init__.py
2. import traceback
4. from lib.conf.config import settings
5. import importlib
6. import subprocess
7. ### 管理插件信息的类
8. class PluginsManager(object):
10. def __init__(self, hostname=None):
11. self.plugins_dict = settings.PLUGINS_DICT
12. self.hostname = hostname
13. self.debug = settings.DEBUG
14. if settings.MODE == 'ssh':
15. self.port = settings.SSH_PORT
16. self.name = settings.SSH_USERNAME
17. self.pwd = settings.SSH_PASSWORD
19. ### 读取配置文件中的pluginsdict， 并执行对应模块中的process方法
20. def execute(self):
21. response = {}
22. for k, v in self.plugins_dict.items():
23. ret = {"status":None, 'data':None}
24. '''
25. k: board,...
26. v: src.plugins.board.Board 字符串
27. '''
28. try:
29. # 1. 导入模块路径
30. moudle_path, class_name = v.rsplit('.', 1)
31. # 2. 导入这个路径
32. moudle_name = importlib.import_module(moudle_path)
33. # 3. 导入对应模块下的类
34. classobj = getattr(moudle_name, class_name)
35. # 4. 执行类下面对应的process方法
36. res = classobj().process(self.__cmd_run, self.debug)
37. ret['status'] = 10000
38. ret['data'] = res
39. except Exception as e:
40. ret['status'] = 10001
41. ret['data']= "[%s] 采集 [%s] 出错了， 错误信息是：%s" % (self.hostname if self.hostname else "Agent", k, str(traceback.format_exc()))
42. response[k] = ret
43. return response
46. def __cmd_run(self, cmd):
47. if settings.MODE == 'agent':
48. return self.__cmd_agent(cmd)
49. elif settings.MODE == 'ssh':
50. return self.__cmd_ssh(cmd)
51. elif settings.MODE == 'salt':
52. return self.__cmd_salt(cmd)
53. else:
54. print("只支持的模式有：agent/ssh/salt")
56. def __cmd_agent(self, cmd):
57. res = subprocess.getoutput(cmd)
58. return res
60. def __cmd_ssh(self, cmd):
61. import paramiko
62. # 创建SSH对象
63. ssh = paramiko.SSHClient()
64. # 允许连接不在know_hosts文件中的主机
65. ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
66. # 连接服务器
67. ssh.connect(hostname=self.hostname, port=self.port, username=self.name, password=self.pwd)
68. # 执行命令
69. stdin, stdout, stderr = ssh.exec_command(cmd)
70. # 获取命令结果
71. result = stdout.read()
72. # 关闭连接
73. ssh.close()
74. return result
76. def __cmd_salt(self, cmd):
77. command = "salt %s cmd.run %s" % (self.hostname, cmd)
78. res = subprocess.getoutput(command)
79. return res

1. # basic.py
3. class Basic(object):
4. def __init__(self):
5. pass
7. @classmethod
8. def initial(cls):
9. return cls()
11. def process(self, command_func, debug):
12. if debug:
13. output = {
14. 'os_platform': "linux",
15. 'os_version': "CentOS release 6.6 (Final)nKernel r on an m",
16. 'hostname': 'c2000.com'
17. }
18. else:
19. output = {
20. 'os_platform': command_func("uname").strip(),
21. 'os_version': command_func("cat /etc/issue").strip().split('n')[0],
22. 'hostname': command_func("hostname").strip(),
23. }
26. return output

1. # board.py
2. import os
3. from lib.conf.config import settings
6. class Board(object):
7. def __init__(self):
8. pass
10. @classmethod
11. def initial(cls):
12. return cls()
14. def process(self, command_func, debug):
15. if debug:
16. output = open(os.path.join(settings.BASEDIR, 'files/board.out'), 'r', encoding='utf-8').read()
17. else:
18. output = command_func("sudo dmidecode -t1")
19. return self.parse(output)
21. def parse(self, content):
23. result = {}
24. key_map = {
25. 'Manufacturer': 'manufacturer',
26. 'Product Name': 'model',
27. 'Serial Number': 'sn',
28. }
30. for item in content.split('n'):
31. row_data = item.strip().split(':')
33. if len(row_data) == 2:
34. if row_data[0] in key_map:
35. result[key_map[row_data[0]]] = row_data[1].strip() if row_data[1] else row_data[1]
37. return result

1. # cpu.py
2. import os
3. from lib.conf.config import settings
5. class Cpu(object):
6. def __init__(self):
7. pass
9. @classmethod
10. def initial(cls):
11. return cls()
13. def process(self, command_func, debug):
14. if debug:
15. output = open(os.path.join(settings.BASEDIR, 'files/cpuinfo.out'), 'r', encoding='utf-8').read()
16. else:
17. output = command_func("cat /proc/cpuinfo")
18. return self.parse(output)
20. def parse(self, content):
21. """
22. 解析shell命令返回结果
23. :param content: shell 命令结果
24. :return:解析后的结果
25. """
26. response = {'cpu_count': 0, 'cpu_physical_count': 0, 'cpu_model': ''}
28. cpu_physical_set = set()
30. content = content.strip()
31. for item in content.split('nn'):
32. for row_line in item.split('n'):
33. key, value = row_line.split(':')
34. key = key.strip()
35. if key == 'processor':
36. response['cpu_count'] += 1
37. elif key == 'physical id':
38. cpu_physical_set.add(value)
39. elif key == 'model name':
40. if not response['cpu_model']:
41. response['cpu_model'] = value
42. response['cpu_physical_count'] = len(cpu_physical_set)
44. return response

1. #disk.py
2. import re
3. import os
4. from lib.conf.config import settings
7. class Disk(object):
8. def __init__(self):
9. pass
11. @classmethod
12. def initial(cls):
13. return cls()
15. def process(self, command_func, debug):
16. if debug:
17. output = open(os.path.join(settings.BASEDIR, 'files/disk.out'), 'r', encoding='utf-8').read()
18. else:
19. output = command_func("sudo MegaCli -PDList -aALL")
20. return self.parse(output)
22. def parse(self, content):
23. """
24. 解析shell命令返回结果
25. :param content: shell 命令结果
26. :return:解析后的结果
27. """
28. response = {}
29. result = []
30. for row_line in content.split("nnnn"):
31. result.append(row_line)
32. for item in result:
33. temp_dict = {}
34. for row in item.split('n'):
35. if not row.strip():
36. continue
37. if len(row.split(':')) != 2:
38. continue
39. key, value = row.split(':')
40. name = self.mega_patter_match(key)
41. if name:
42. if key == 'Raw Size':
43. raw_size = re.search('(d+.d+)', value.strip())
44. if raw_size:
46. temp_dict[name] = raw_size.group()
47. else:
48. raw_size = '0'
49. else:
50. temp_dict[name] = value.strip()
51. if temp_dict:
52. response[temp_dict['slot']] = temp_dict
53. return response
55. @staticmethod
56. def mega_patter_match(needle):
57. grep_pattern = {'Slot': 'slot', 'Raw Size': 'capacity', 'Inquiry': 'model', 'PD Type': 'pd_type'}
58. for key, value in grep_pattern.items():
59. if needle.startswith(key):
60. return value
61. return False

1. # memory.py
2. import os
3. from lib import convert
4. from lib.conf.config import settings
7. class Memory(object):
8. def __init__(self):
9. pass
11. @classmethod
12. def initial(cls):
13. return cls()
15. def process(self, command_func, debug):
16. if debug:
17. output = open(os.path.join(settings.BASEDIR, 'files/memory.out'), 'r', encoding='utf-8').read()
18. else:
19. output = command_func("sudo dmidecode -q -t 17 2>/dev/null")
21. return self.parse(output)
23. def parse(self, content):
24. """
25. 解析shell命令返回结果
26. :param content: shell 命令结果
27. :return:解析后的结果
28. """
29. ram_dict = {}
30. key_map = {
31. 'Size': 'capacity',
32. 'Locator': 'slot',
33. 'Type': 'model',
34. 'Speed': 'speed',
35. 'Manufacturer': 'manufacturer',
36. 'Serial Number': 'sn',
38. }
39. devices = content.split('Memory Device')
40. for item in devices:
41. item = item.strip()
42. if not item:
43. continue
44. if item.startswith('#'):
45. continue
46. segment = {}
47. lines = item.split('nt')
48. for line in lines:
49. if not line.strip():
50. continue
51. if len(line.split(':')):
52. key, value = line.split(':')
53. else:
54. key = line.split(':')[0]
55. value = ""
56. if key in key_map:
57. if key == 'Size':
58. segment[key_map['Size']] = convert.convert_mb_to_gb(value, 0)
59. else:
60. segment[key_map[key.strip()]] = value.strip()
62. ram_dict[segment['slot']] = segment
64. return ram_dict

1. #nic.py 网络接口控制器
2. import os
3. import re
4. from lib.conf.config import settings
6. class Nic(object):
7. def __init__(self):
8. pass
10. @classmethod
11. def initial(cls):
12. return cls()
14. def process(self, command_func, debug):
15. if debug:
16. output = open(os.path.join(settings.BASEDIR, 'files/nic.out'), 'r', encoding='utf-8').read()
17. interfaces_info = self._interfaces_ip(output)
18. else:
19. interfaces_info = self.linux_interfaces(command_func)
21. self.standard(interfaces_info)
23. return interfaces_info
25. def linux_interfaces(self, command_func):
26. '''
27. Obtain interface information for *NIX/BSD variants
28. '''
29. ifaces = dict()
30. ip_path = 'ip'
31. if ip_path:
32. cmd1 = command_func('sudo {0} link show'.format(ip_path))
33. cmd2 = command_func('sudo {0} addr show'.format(ip_path))
34. ifaces = self._interfaces_ip(cmd1 + 'n' + cmd2)
35. return ifaces
37. def which(self, exe):
38. def _is_executable_file_or_link(exe):
39. # check for os.X_OK doesn't suffice because directory may executable
40. return (os.access(exe, os.X_OK) and
41. (os.path.isfile(exe) or os.path.islink(exe)))
43. if exe:
44. if _is_executable_file_or_link(exe):
45. # executable in cwd or fullpath
46. return exe
48. # default path based on busybox's default
49. default_path = '/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin'
50. search_path = os.environ.get('PATH', default_path)
51. path_ext = os.environ.get('PATHEXT', '.EXE')
52. ext_list = path_ext.split(';')
54. search_path = search_path.split(os.pathsep)
55. if True:
56. # Add any dirs in the default_path which are not in search_path. If
57. # there was no PATH variable found in os.environ, then this will be
58. # a no-op. This ensures that all dirs in the default_path are
59. # searched, which lets salt.utils.which() work well when invoked by
60. # salt-call running from cron (which, depending on platform, may
61. # have a severely limited PATH).
62. search_path.extend(
63. [
64. x for x in default_path.split(os.pathsep)
65. if x not in search_path
66. ]
67. )
68. for path in search_path:
69. full_path = os.path.join(path, exe)
70. if _is_executable_file_or_link(full_path):
71. return full_path
73. return None
75. def _number_of_set_bits_to_ipv4_netmask(self, set_bits): # pylint: disable=C0103
76. '''
77. Returns an IPv4 netmask from the integer representation of that mask.
79. Ex. 0xffffff00 -> '255.255.255.0'
80. '''
81. return self.cidr_to_ipv4_netmask(self._number_of_set_bits(set_bits))
83. def cidr_to_ipv4_netmask(self, cidr_bits):
84. '''
85. Returns an IPv4 netmask
86. '''
87. try:
88. cidr_bits = int(cidr_bits)
89. if not 1 <= cidr_bits <= 32:
90. return ''
91. except ValueError:
92. return ''
94. netmask = ''
95. for idx in range(4):
96. if idx:
97. netmask += '.'
98. if cidr_bits >= 8:
99. netmask += '255'
100. cidr_bits -= 8
101. else:
102. netmask += '{0:d}'.format(256 - (2 ** (8 - cidr_bits)))
103. cidr_bits = 0
104. return netmask
106. def _number_of_set_bits(self, x):
107. '''
108. Returns the number of bits that are set in a 32bit int
109. '''
110. # Taken from http://stackoverflow.com/a/4912729. Many thanks!
111. x -= (x >> 1) & 0x55555555
112. x = ((x >> 2) & 0x33333333) + (x & 0x33333333)
113. x = ((x >> 4) + x) & 0x0f0f0f0f
114. x += x >> 8
115. x += x >> 16
116. return x & 0x0000003f
118. def _interfaces_ip(self, out):
119. '''
120. Uses ip to return a dictionary of interfaces with various information about
121. each (up/down state, ip address, netmask, and hwaddr)
122. '''
123. ret = dict()
124. right_keys = ['name', 'hwaddr', 'up', 'netmask', 'ipaddrs']
126. def parse_network(value, cols):
127. '''
128. Return a tuple of ip, netmask, broadcast
129. based on the current set of cols
130. '''
131. brd = None
132. if '/' in value: # we have a CIDR in this address
133. ip, cidr = value.split('/') # pylint: disable=C0103
134. else:
135. ip = value # pylint: disable=C0103
136. cidr = 32
138. if type_ == 'inet':
139. mask = self.cidr_to_ipv4_netmask(int(cidr))
140. if 'brd' in cols:
141. brd = cols[cols.index('brd') + 1]
142. return (ip, mask, brd)
144. groups = re.compile('r?n\d').split(out)
145. for group in groups:
146. iface = None
147. data = dict()
149. for line in group.splitlines():
150. if ' ' not in line:
151. continue
152. match = re.match(r'^d*:s+([w.-]+)(?:@)?([w.-]+)?:s+<(.+)>', line)
153. if match:
154. iface, parent, attrs = match.groups()
155. if 'UP' in attrs.split(','):
156. data['up'] = True
157. else:
158. data['up'] = False
159. if parent and parent in right_keys:
160. data[parent] = parent
161. continue
163. cols = line.split()
164. if len(cols) >= 2:
165. type_, value = tuple(cols[0:2])
167. iflabel = cols[-1:][0]
168. if type_ in ('inet',):
169. if 'secondary' not in cols:
170. ipaddr, netmask, broadcast = parse_network(value, cols)
171. if type_ == 'inet':
172. if 'inet' not in data:
173. data['inet'] = list()
174. addr_obj = dict()
175. addr_obj['address'] = ipaddr
176. addr_obj['netmask'] = netmask
177. addr_obj['broadcast'] = broadcast
178. data['inet'].append(addr_obj)
179. else:
180. if 'secondary' not in data:
181. data['secondary'] = list()
182. ip_, mask, brd = parse_network(value, cols)
183. data['secondary'].append({
184. 'type': type_,
185. 'address': ip_,
186. 'netmask': mask,
187. 'broadcast': brd,
188. })
189. del ip_, mask, brd
190. elif type_.startswith('link'):
191. data['hwaddr'] = value
192. if iface:
193. if iface.startswith('pan') or iface.startswith('lo') or iface.startswith('v'):
194. del iface, data
195. else:
196. ret[iface] = data
197. del iface, data
198. return ret
200. def standard(self, interfaces_info):
202. for key, value in interfaces_info.items():
203. ipaddrs = set()
204. netmask = set()
205. if not 'inet' in value:
206. value['ipaddrs'] = ''
207. value['netmask'] = ''
208. else:
209. for item in value['inet']:
210. ipaddrs.add(item['address'])
211. netmask.add(item['netmask'])
212. value['ipaddrs'] = '/'.join(ipaddrs)
213. value['netmask'] = '/'.join(netmask)
214. del value['inet']

1. # lib/convert.py
2. def convert_to_int(value,default=0):
4. try:
5. result = int(value)
6. except Exception as e:
7. result = default
9. return result
11. def convert_mb_to_gb(value,default=0):
13. try:
14. value = value.strip('MB')
15. result = int(value)
16. except Exception as e:
17. result = default
19. return result

1. # bin/start.py
2. from src.plugins import PluginsManager
3. if __name__ == '__main__':
4. res=PluginsManager().execute()
5. print(res)

注意

sudo dmidecode -t1

可以获取BIOS，系统，主板，处理器，内存，缓存等 序列号、电脑厂商、串口信息以及其它系统配件信息

https://ipcmen.com/dmidecode

sudo MegaCli -PDList -aALL

需要安装

2.7 异常处理

traceback使用

1. import traceback
2. def test():
3. try:
4. a = "dsadsa"
5. int(a)
6. except Exception as e:
7. print(traceback.format_exc())
9. test()

src/plugins/init.py

1. import traceback
3. from lib.conf.config import settings
4. import importlib
5. import subprocess
6. ### 管理插件信息的类
7. class PluginsManager(object):
9. def __init__(self, hostname=None):
10. self.plugins_dict = settings.PLUGINS_DICT
11. self.hostname = hostname
12. self.debug = settings.DEBUG
13. if settings.MODE == 'ssh':
14. self.port = settings.SSH_PORT
15. self.name = settings.SSH_USERNAME
16. self.pwd = settings.SSH_PASSWORD
18. ### 读取配置文件中的pluginsdict， 并执行对应模块中的process方法
19. def execute(self):
20. response = {}
21. for k, v in self.plugins_dict.items():
22. ret = {"status":None, 'data':None}
23. '''
24. k: board,...
25. v: src.plugins.board.Board 字符串
26. '''
27. try:
28. # 1. 导入模块路径
29. moudle_path, class_name = v.rsplit('.', 1)
30. # 2. 导入这个路径
31. moudle_name = importlib.import_module(moudle_path)
32. # 3. 导入对应模块下的类
33. classobj = getattr(moudle_name, class_name)
34. # 4. 执行类下面对应的process方法
35. res = classobj().process(self.__cmd_run, self.debug)
36. ret['status'] = 10000
37. ret['data'] = res
38. except Exception as e:
39. # hostname有值说明不是anget方案，是salstack或paramiko方案
40. ret['status'] = 10001
41. ret['data']= "[%s] 采集 [%s] 出错了， 错误信息是：%s" % (self.hostname if self.hostname else "Agent", k, str(traceback.format_exc()))
42. response[k] = ret
43. return response
46. def __cmd_run(self, cmd):
47. if settings.MODE == 'agent':
48. return self.__cmd_agent(cmd)
49. elif settings.MODE == 'ssh':
50. return self.__cmd_ssh(cmd)
51. elif settings.MODE == 'salt':
52. return self.__cmd_salt(cmd)
53. else:
54. print("只支持的模式有：agent/ssh/salt")
56. def __cmd_agent(self, cmd):
57. res = subprocess.getoutput(cmd)
58. return res
60. def __cmd_ssh(self, cmd):
61. import paramiko
62. # 创建SSH对象
63. ssh = paramiko.SSHClient()
64. # 允许连接不在know_hosts文件中的主机
65. ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
66. # 连接服务器
67. ssh.connect(hostname=self.hostname, port=self.port, username=self.name, password=self.pwd)
68. # 执行命令
69. stdin, stdout, stderr = ssh.exec_command(cmd)
70. # 获取命令结果
71. result = stdout.read()
72. # 关闭连接
73. ssh.close()
74. return result
76. def __cmd_salt(self, cmd):
77. command = "salt %s cmd.run %s" % (self.hostname, cmd)
78. res = subprocess.getoutput(command)
79. return res

2.8 把采集到的数据上传

客户端

1. ## src/client
2. import requests
3. from lib.conf.config import settings
4. from src.plugins import PluginsManager
5. import os
6. class Base():
7. def post_data(self, server_info):
8. requests.post(settings.API_URL, json=server_info)
10. class Agent(Base):
11. ### 收集数据并发送
12. def collectAndPost(self):
13. server_info = PluginsManager().execute()
15. hostname = server_info['basic']['data']['hostname'] ### c10000.com
16. res = open(os.path.join(settings.BASEDIR, 'config/cert'), 'r', encoding='utf-8').read()
18. if not res.strip():
19. #### 第一次采集, 将采集的hostname写入到一个文件中
20. with open(os.path.join(settings.BASEDIR, 'config/cert'), 'w', encoding='utf-8') as fp:
21. fp.write(hostname)
22. else:
23. #### 第二次采集的时候， 永远以第一次文件中保存的主机名为标准
24. server_info['basic']['data']['hostname'] = res
27. for k, v in server_info.items():
28. print(k, v)
29. # requests.post(settings.API_URL, data=json.dumps(res))
30. ### Content-Type':"application/json"
31. self.post_data(server_info)
34. class SSHSalt(Base):
35. def get_hostnames(self):
36. hostnames = requests.get(settings.API_URL)
37. return ['c1.com', 'c2.com']
39. def run(self, hostname):
40. server_info = PluginsManager(hostname).execute()
41. self.post_data(server_info)
43. def collectAndPost(self):
44. hostnames = self.get_hostnames()
45. ### 单线程执行， 循环速度比较慢
46. # for hostname in hostnames:
47. # server_info = PluginsManager(hostname).execute()
48. # self.post_data(server_info)
50. ### 线程池的方式采集数据
51. from concurrent.futures import ThreadPoolExecutor
52. p = ThreadPoolExecutor(10)
53. for hostname in hostnames:
54. p.submit(self.run, hostname)

1. # src/script.py
2. from src.client import Agent
3. from src.client import SSHSalt
4. from lib.conf.config import settings
6. def run():
7. if settings.MODE == 'agent':
8. obj = Agent()
9. else:# 不管salt和paramiko方式，都需要从服务器获取客户端ip地址
10. obj = SSHSalt()
11. obj.collectAndPost()

1. # bin/start.py
2. from src.script import run
3. if __name__ == '__main__':
4. run()

服务端

2.9 唯一标识的问题

2. # 目标：将变更的信息通过程序的比对， 记录下来
3. #第一天的时候：
4. # 采集数据：
5. {'status': 10000, 'data': {'os_platform': 'linux', 'os_version': 'CentOS release 6.6 (Final)nKernel r on an \m', 'hostname': 'c2.com'}}
7. #API清洗的时候:
8. 因为是第一次， 数据库中并没有采集的数据
9. 数据入库：
10. server：1000条
11. id sn os_platform os_version disk_size
12. 1 dsadsa linux CentOS 250G
13. ........
14. #第二天的时候(数据发生变化，应该比对)：
15. #采集数据：
16. {'status': 10000, 'data': {'os_platform': 'linux', 'os_version': 'CentOS release 6.6 (Final)nKernel r on an \m', 'hostname': 'c2.com'}}
18. {'status': 10000, 'data': {'0': {'slot': '0', 'pd_type': 'SAS', 'capacity': '300G', 'model': 'SEAGATE ST300MM0006 LS08S0K2B5NV'}, '1': {'slot': '1', 'pd_type': 'SAS', 'capacity': '279.396', 'model': 'SEAGATE ST300MM0006 LS08S0K2B5AH'}, '2': {'slot': '2', 'pd_type': 'SATA', 'capacity': '476.939', 'model': 'S1SZNSAFA01085L Samsung SSD 850 PRO 512GB EXM01B6Q'}, '3': {'slot': '3', 'pd_type': 'SATA', 'capacity': '476.939', 'model': 'S1AXNSAF912433K Samsung SSD 840 PRO Series DXM06B0Q'}, '4': {'slot': '4', 'pd_type': 'SATA', 'capacity': '476.939', 'model': 'S1AXNSAF303909M Samsung SSD 840 PRO Series DXM05B0Q'}, '5': {'slot': '5', 'pd_type': 'SATA', 'capacity': '476.939', 'model': 'S1AXNSAFB00549A Samsung SSD 840 PRO Series DXM06B0Q'}}}
20. # API清洗的时候:
21. 应该在新的POST数据中选取一个 唯一 的字段， 然后到数据库中作为where条件， 获取到对应的数据
23. 问题是 应该选取谁？
24. 选取的是 sn 序列号（mac地址） 作为唯一的字段
25. 用sn遇到的问题：
26. 虚拟机和实体机共用一个sn， 导致数据不准确
28. # 解决的方案：
29. a. 如果公司不需要采集虚拟机的信息， 使用sn没有问题
30. b. 采用 hostname 作为唯一标识
31. - 是允许开发可以临时修改主机名的
32. -实现方案：
33. -1. 给这些服务器分配唯一的主机名
34. -2 将分配好的主机名录入到后台管理的DBserver表中
35. -3. 将采集的client客户端代码， 运行一次
36. -4 然后将得到的主机名地址保存到一个文件中
37. 第一天：
38. 1. 给这些服务器分配唯一的主机名
39. 2. 将分配好的主机名录入到后台管理的DBserver表中
40. 3. 将采集的client客户端代码， 运行一次，
41. 然后将得到的主机名地址保存到一个文件中
42. 第二天：
43. hostname = server_info['basic']['data']['hostname'] ### c10000.com
44. res = open(os.path.join(settings.BASEDIR, 'config/cert'), 'r', encoding='utf-8').read()
45. if not res.strip():
46. #### 第一次采集, 将采集的hostname写入到一个文件中
47. with open(os.path.join(settings.BASEDIR, 'config/cert'), 'w', encoding='utf-8') as fp:
48. fp.write(hostname)
49. else:
50. #### 第二次采集的时候， 永远以第一次文件中保存的主机名为标准
51. server_info['basic']['data']['hostname'] = res

代码实现

1. # src/client.py
2. # angent方案：第一次运行时取主机名，写到文件中，以后永远用主机名
3. # ssh和salt方案，不需要此操作，因为一旦主机名改了，就连接不上了
4. import requests
5. from lib.conf.config import settings
6. from src.plugins import PluginsManager
7. import os
8. class Base():
9. def post_data(self, server_info):
10. requests.post(settings.API_URL, json=server_info)
12. class Agent(Base):
13. ### 收集数据并发送
14. def collectAndPost(self):
15. server_info = PluginsManager().execute()
17. hostname = server_info['basic']['data']['hostname'] ### c10000.com
18. res = open(os.path.join(settings.BASEDIR, 'config/cert'), 'r', encoding='utf-8').read()
20. if not res.strip():
21. #### 第一次采集, 将采集的hostname写入到一个文件中
22. with open(os.path.join(settings.BASEDIR, 'config/cert'), 'w', encoding='utf-8') as fp:
23. fp.write(hostname)
24. else:
25. #### 第二次采集的时候， 永远以第一次文件中保存的主机名为标准
26. server_info['basic']['data']['hostname'] = res
29. for k, v in server_info.items():
30. print(k, v)
31. # requests.post(settings.API_URL, data=json.dumps(res))
32. ### Content-Type':"application/json"
33. self.post_data(server_info)
36. class SSHSalt(Base):
37. pass

2.10 API的验证

第一种方式

1. # 客户端：
3. #### 第一种方式
4. import requests
5. token = "dsabdshanbdjsanjdsanjds"
6. #### 切记， 进行token验证的时候， 一定是将token写在http的请求头中
7. res = requests.get("http://127.0.0.1:8000/getInfo/", headers = {"token":token})
8. print(res.text)
10. # 服务端：
12. token = request.META.get('HTTP_TOKEN')
13. server_token = "dsabdshanbdjsanjdsanjdsa"
14. if token != server_token:
15. return HttpResponse('token值是错误的！')

第二种方式

1. ## 客户端：
2. import requests
3. token = "dsabdshanbdjsanjdsanjds"
5. import time
6. client_time = time.time()
7. tmp = "%s|%s" % (token, client_time)
9. ##### 加密
10. import hashlib
11. m = hashlib.md5()
12. m.update(bytes(tmp, encoding='utf8'))
13. res = m.hexdigest()
14. client_md5_token = "%s|%s" % (res, client_time)
16. #### 切记， 进行token验证的时候， 一定是将token写在http的请求头中
17. data = requests.get("http://127.0.0.1:8000/getInfo/", headers = {"token":client_md5_token})
18. print(data.text)
21. # 服务端：
22. server_token = "dsabdshanbdjsanjdsanjds"
23. server_time = time.time()
24. client_md5_header = request.META.get('HTTP_TOKEN')
25. client_md5_token, client_time = client_md5_header.split('|')
26. client_time = float(client_time)
28. if server_time - client_time > 10:
29. return HttpResponse(' 时间太久了.....')
31. tmp = "%s|%s" % (server_token, client_time)
32. m = hashlib.md5()
33. m.update(bytes(tmp, encoding='utf-8'))
34. server_md5_token = m.hexdigest()
36. if server_md5_token != client_md5_token:
37. return HttpResponse('修改了token')

三 服务端编写

3.1 后台表结构

Disk表:

NIC表：

Memory表：

Server表：机器位置信息，在哪个机房，机房基层，机柜位置，部署时间这些属性手动录入

跟上面三个表是一对多

IDC表：机房表，跟Server是一对多

BusinessUnit表：业务线（产品线）表，跟server是一对多

Tag表：标签表，跟Server是多对多

UserInfo表：用户表，分产品线表是多对多

UserGroup表：用户组表，跟用户多对多

AssetRecord表：资产变更记录表，server跟AssetRecord是一对多

ErrorLog表：错误日志表，server跟errorlog是一对多

1. from django.db import models
3. class UserProfile(models.Model):
4. """
5. 用户信息
6. """
7. name = models.CharField(u'姓名', max_length=32)
8. email = models.EmailField(u'邮箱')
9. phone = models.CharField(u'座机', max_length=32)
10. mobile = models.CharField(u'手机', max_length=32)
11. password = models.CharField(u'密码', max_length=64)
13. class Meta:
14. verbose_name_plural = "用户表"
16. def __str__(self):
17. return self.name
21. class UserGroup(models.Model):
22. """
23. 用户组
24. """
25. name = models.CharField(max_length=32, unique=True)
26. users = models.ManyToManyField('UserProfile')
28. class Meta:
29. verbose_name_plural = "用户组表"
31. def __str__(self):
32. return self.name
35. class BusinessUnit(models.Model):
36. """
37. 业务线
38. """
39. name = models.CharField('业务线', max_length=64, unique=True)
40. contact = models.ForeignKey('UserGroup', verbose_name='业务联系人', related_name='c')
41. manager = models.ForeignKey('UserGroup', verbose_name='系统管理员', related_name='m')
43. class Meta:
44. verbose_name_plural = "业务线表"
46. def __str__(self):
47. return self.name
50. class IDC(models.Model):
51. """
52. 机房信息
53. """
54. name = models.CharField('机房', max_length=32)
55. floor = models.IntegerField('楼层', default=1)
57. class Meta:
58. verbose_name_plural = "机房表"
60. def __str__(self):
61. return self.name
64. class Tag(models.Model):
65. """
66. 资产标签
67. """
68. name = models.CharField('标签', max_length=32, unique=True)
70. class Meta:
71. verbose_name_plural = "标签表"
73. def __str__(self):
74. return self.name
78. class Server(models.Model):
79. """
80. 服务器信息
81. """
82. device_type_choices = (
83. (1, '服务器'),
84. (2, '交换机'),
85. (3, '防火墙'),
86. )
87. device_status_choices = (
88. (1, '上架'),
89. (2, '在线'),
90. (3, '离线'),
91. (4, '下架'),
92. )
94. device_type_id = models.IntegerField('服务器类型',choices=device_type_choices, default=1)
95. device_status_id = models.IntegerField('服务器状态',choices=device_status_choices, default=1)
97. cabinet_num = models.CharField('机柜号', max_length=30, null=True, blank=True)
98. cabinet_order = models.CharField('机柜中序号', max_length=30, null=True, blank=True)
100. idc = models.ForeignKey('IDC', verbose_name='IDC机房', null=True, blank=True)
101. business_unit = models.ForeignKey('BusinessUnit', verbose_name='属于的业务线', null=True, blank=True)
103. tag = models.ManyToManyField('Tag')
105. hostname = models.CharField('主机名',max_length=128, unique=True)
106. sn = models.CharField('SN号', max_length=64, db_index=True)
107. manufacturer = models.CharField(verbose_name='制造商', max_length=64, null=True, blank=True)
108. model = models.CharField('型号', max_length=64, null=True, blank=True)
110. manage_ip = models.GenericIPAddressField('管理IP', null=True, blank=True)
112. os_platform = models.CharField('系统', max_length=16, null=True, blank=True)
113. os_version = models.CharField('系统版本', max_length=16, null=True, blank=True)
115. cpu_count = models.IntegerField('CPU个数', null=True, blank=True)
116. cpu_physical_count = models.IntegerField('CPU物理个数', null=True, blank=True)
117. cpu_model = models.CharField('CPU型号', max_length=128, null=True, blank=True)
119. create_at = models.DateTimeField(auto_now_add=True, blank=True)
121. class Meta:
122. verbose_name_plural = "服务器表"
124. def __str__(self):
125. return self.hostname
128. class Disk(models.Model):
129. """
130. 硬盘信息
131. """
132. slot = models.CharField('插槽位', max_length=8)
133. model = models.CharField('磁盘型号', max_length=32)
134. capacity = models.CharField('磁盘容量GB', max_length=32)
135. pd_type = models.CharField('磁盘类型', max_length=32)
136. server_obj = models.ForeignKey('Server',related_name='disk')
138. class Meta:
139. verbose_name_plural = "硬盘表"
141. def __str__(self):
142. return self.slot
145. class NIC(models.Model):
146. """
147. 网卡信息
148. """
149. name = models.CharField('网卡名称', max_length=128)
150. hwaddr = models.CharField('网卡mac地址', max_length=64)
151. netmask = models.CharField(max_length=64)
152. ipaddrs = models.CharField('ip地址', max_length=256)
153. up = models.BooleanField(default=False)
154. server_obj = models.ForeignKey('Server',related_name='nic')
157. class Meta:
158. verbose_name_plural = "网卡表"
160. def __str__(self):
161. return self.name
164. class Memory(models.Model):
165. """
166. 内存信息
167. """
168. slot = models.CharField('插槽位', max_length=32)
169. manufacturer = models.CharField('制造商', max_length=32, null=True, blank=True)
170. model = models.CharField('型号', max_length=64)
171. capacity = models.FloatField('容量', null=True, blank=True)
172. sn = models.CharField('内存SN号', max_length=64, null=True, blank=True)
173. speed = models.CharField('速度', max_length=16, null=True, blank=True)
175. server_obj = models.ForeignKey('Server',related_name='memory')
178. class Meta:
179. verbose_name_plural = "内存表"
181. def __str__(self):
182. return self.slot
184. class AssetRecord(models.Model):
185. """
186. 资产变更记录,creator为空时，表示是资产汇报的数据。
187. """
188. asset_obj = models.ForeignKey('Server', related_name='ar')
189. content = models.TextField(null=True)# 新增硬盘
190. creator = models.ForeignKey('UserProfile', null=True, blank=True) #
191. create_at = models.DateTimeField(auto_now_add=True)
194. class Meta:
195. verbose_name_plural = "资产记录表"
197. def __str__(self):
198. return "%s-%s-%s" % (self.asset_obj.idc.name, self.asset_obj.cabinet_num, self.asset_obj.cabinet_order)
201. class ErrorLog(models.Model):
202. """
203. 错误日志,如：agent采集数据错误 或 运行错误
204. """
205. asset_obj = models.ForeignKey('Server', null=True, blank=True)
206. title = models.CharField(max_length=16)
207. content = models.TextField()
208. create_at = models.DateTimeField(auto_now_add=True)
210. class Meta:
211. verbose_name_plural = "错误日志表"
213. def __str__(self):
214. return self.title
216. # admin管理
217. from repository import models
218. admin.site.register(models.Server)
219. admin.site.register(models.UserProfile)
220. admin.site.register(models.UserGroup)
221. admin.site.register(models.BusinessUnit)
222. admin.site.register(models.IDC)
223. admin.site.register(models.Tag)
224. admin.site.register(models.Disk)
225. admin.site.register(models.Memory)
226. admin.site.register(models.NIC)
227. admin.site.register(models.AssetRecord)
228. admin.site.register(models.ErrorLog)
230. # 录入信息
231. # 录入三条业务线：互娱部，新闻部，云计算部
232. # 录入用户组：A组，B组，C组
233. # 录入管理员：张三，李四，王五
234. # 录入Server数据：服务器，上架，机柜号13，机柜中序号32，IDC机房，业务线，标签，主机名（c2.com）
235. # 录入IDC机房：世纪互联，神州
236. # 录入标签：web，db，cache

3.2 资产清洗录入（以硬盘为例）

1. # 新增：new-old
2. # 删除：old-new
3. # 更新：交集
5. # 差集
6. new_slot_list={0,1,2}
7. old_slot_list={0,1}
8. # 差集
9. res=new_slot_list-old_slot_list
10. print(res)
11. # 或者
12. res=new_slot_list.difference(old_slot_list)
13. print(res)
15. # 交集
16. print(new_slot_list & old_slot_list)
17. # 或者
18. print(new_slot_list.intersection(old_slot_list))

1. def getInfo(request):
3. if request.method == 'POST':
4. data = request.body
5. # print(data)
6. data = json.loads(data)
8. #### 通过主机名获取老的数据对应的记录
9. hostname = data['basic']['data']['hostname']
11. old_server_info = models.Server.objects.filter(hostname=hostname).first() ## obj
13. if not old_server_info:
14. return HttpResponse('资产不存在')
17. #### 以分析disk硬盘数据为例， 进行比对分析
19. #### 如果采集出错的话， 记录错误的信息
20. if data['disk']['status'] != 10000:
21. models.ErrorLog.objects.create(asset_obj=old_server_info, title = "%s 采集硬盘出错了" % (hostname), content=data['disk']['data'])
23. '''
24. {
25. '0': {'slot': '0', 'pd_type': 'SAS', 'capacity': '279.396', 'model': 'SEAGATE ST300MM0006 LS08S0K2B5NV'},
26. '1': {'slot': '1', 'pd_type': 'SAS', 'capacity': '279.396', 'model': 'SEAGATE ST300MM0006 LS08S0K2B5AH'},
27. '2': {'slot': '2', 'pd_type': 'SATA', 'capacity': '476.939', 'model': 'S1SZNSAFA01085L Samsung SSD 850 PRO 512GB EXM01B6Q'},
28. }
29. '''
30. new_disk_info = data['disk']['data']
32. '''
33. [
34. obj(slot:0, pd_type:SAS,......),
35. obj(slot:1, pd_type:SATA,......),
36. ....
37. ]
38. '''
39. old_disk_info = models.Disk.objects.filter(server_obj=old_server_info).all() ## []
41. new_slot_list = list(new_disk_info.keys())
42. old_slot_list = []
43. for obj in old_disk_info:
44. old_slot_list.append(obj.slot)
46. '''
47. new_slot_list = [0,2]
48. old_slot_list = [0,1]
49. 新增： new_slot_list - old_slot_list = 2
50. 删除： old_slot_list - new_slot_list = 1
51. 更新： 交集
52. '''
53. #### 增加slot
54. add_slot_list = set(new_slot_list).difference(set(old_slot_list))
56. if add_slot_list:
57. record_list = []
58. for slot in add_slot_list:
59. # {'slot': '0', 'pd_type': 'SAS', 'capacity': '279.396', 'model': 'SEAGATE ST300MM0006 LS08S0K2B5NV'}
60. disk_res = new_disk_info[slot]
61. tmp = "添加插槽是:{slot}, 磁盘类型是:{pd_type}, 磁盘容量是:{capacity}, 磁盘的型号：{model}".format(**disk_res)
62. disk_res['server_obj'] = old_server_info
63. record_list.append(tmp)
64. models.Disk.objects.create(**disk_res)
66. ### 将变更新的信息添加到变更记录表中
67. record_str = ";".join(record_list)
68. models.AssetRecord.objects.create(asset_obj=old_server_info, content=record_str)
70. #### 删除slot
71. del_slot_list = set(old_slot_list).difference(set(new_slot_list))
72. if del_slot_list:
73. record_str = "删除的槽位是：%s" % (";".join(del_slot_list))
74. models.Disk.objects.filter(slot__in=del_slot_list, server_obj=old_server_info).delete()
75. models.AssetRecord.objects.create(asset_obj=old_server_info, content=record_str)
78. #### 更新硬盘数据
79. up_solt_list = set(new_slot_list).intersection(set(old_slot_list))
80. if up_solt_list:
81. record_list = []
82. for slot in up_solt_list:
83. ## 新的：'0': {'slot': '0', 'pd_type': 'SAS', 'capacity': '500G', 'model': 'SEAGATE ST300MM0006 LS08S0K2B5NV'}
84. new_disk_row = new_disk_info[slot]
85. ### 老的：obj(slot:0, pd_type:SAS,.....)
86. old_disk_row = models.Disk.objects.filter(slot=slot, server_obj=old_server_info).first()
87. for k, new_v in new_disk_row.items():
88. '''
89. k: slot, pd_type, capacity,...
90. new_v: 0 SAS 279.396,....
91. '''
92. ### 利用反射
93. ### 1. 先从老的数据中心获取老的数据
94. old_v = getattr(old_disk_row, k)
95. ### 2. 判断老的数据和新的数据是否相同
96. if new_v != old_v:
97. tmp = "槽位%s, %s由原来的%s变成了%s" % (slot, k, old_v, new_v)
98. record_list.append(tmp)
99. ### 3. 将新的数据设置回到老的数据行对象中
100. setattr(old_disk_row, k, new_v)
101. ### 4. 调用save， 保存
102. old_disk_row.save()
104. if record_list:
105. models.AssetRecord.objects.create(asset_obj=old_server_info, content=";".join(record_list))
108. return HttpResponse('ok')
109. else:
110. ### 第一种方式的判断
111. # if token != server_token:
112. # return HttpResponse('token值是错误的！')
114. ### 连接数据库获取主机名列表
115. token = request.META.get('HTTP_TOKEN')
116. client_md5_token, client_time = token.split('|')
118. client_time = float(client_time)
120. import time
121. server_time = time.time()
122. if server_time - client_time > 10:
123. return HttpResponse('第一关【超时了】')
125. server_token = "dsabdshanbdjsanjdsanjdsa"
127. tmp = "%s|%s" % (server_token, client_time)
128. import hashlib
129. m = hashlib.md5()
130. m.update(bytes(tmp, encoding='utf8'))
131. server_md5_token = m.hexdigest()
132. if server_md5_token != client_md5_token:
133. return HttpResponse('第二关【数据被修改过了】')
136. #### 第三关， 连接redis
138. ### 第一次来的时候， 先去redis中判断， client_md5_token 是否在redis中，
139. ### 如果在redis中， 则代表已经访问过了， return 回去
140. ### 如果不在redis中， 则第一次访问， 添加到redis中， 并且设置过期时间 10s
143. return HttpResponse('非常重要的数据')

3.3 前后端混合开发之layui

1. https://www.layui.com/doc/element/layout.html#adminhttps://www.layui.com/doc/element/layout.html#admin

3.4 前后端混合开发之xadmin

1. # adminx.py
2. import xadmin
4. from repository import models
7. class DiskAdmin(object):
8. list_display = ['id','slot' ,'model','capacity','pd_type','server_obj']
10. search_fields = ['id', 'slot' ,'model','capacity','pd_type']
11. # list_editable = ['name' ,'email','phone','mobile']
12. # list_filter = ['name' ,'email','phone','mobile']
13. # list_filter = ['oid','user' ,'odate','oisPay','ototal','oadress']
15. class ServerAdmin(object):
16. list_display = ['id', 'device_type_id', 'device_status_id', 'idc', 'business_unit', 'hostname', 'create_at']
18. show_detail_fields = ['hostname']
19. # search_fields = ['id', 'slot', 'model', 'capacity', 'pd_type']
20. # data_charts = {
21. # "user_count": {'title': u"服务器分布", "x-field": "idc", "y-field": ("business_unit",),},
22. # # "avg_count": {'title': u"Avg Report", "x-field": "date", "y-field": ('avg_count',), "order": ('date',)}
23. # }
24. # list_per_page = 2
25. data_charts = {
26. "host_service_type_counts": {
27. 'title': '部门机器使用情况',
28. 'x-field': "business_unit",
29. 'y-field': ("business_unit"),
30. 'option': {
31. "series": {"bars": {"align": "center", "barWidth": 0.8, "show": True}},
32. "xaxis": {"aggregate": "count", "mode": "categories"}
33. },
34. },
35. "host_idc_counts": {
36. 'title': '机房统计',
37. 'x-field': "idc",
38. 'y-field': ("idc",),
39. 'option': {
40. "series": {"bars": {"align": "center", "barWidth": 0.3, "show": True}},
41. "xaxis": {"aggregate": "count", "mode": "categories"}
42. }
43. }
44. }
47. class IDCAdmin(object):
48. list_display = ['id', 'name', 'floor']
50. show_detail_fields = ['name']
51. # search_fields = ['id', 'slot', 'model', 'capacity', 'pd_type']
54. xadmin.site.register(models.Disk,DiskAdmin)
55. xadmin.site.register(models.Server,ServerAdmin)
56. xadmin.site.register(models.IDC,IDCAdmin)

3.5 前后端分离之vue-admin

1. # 介绍地址
2. https://panjiachen.github.io/vue-element-admin-site/zh/guide/
4. # 集成版本（高级版本）
5. https://github.com/PanJiaChen/vue-element-admin
6. # 演示地址
7. https://github.com/PanJiaChen/vue-element-admin/blob/master/README.zh-CN.md
9. # 基础版本
10. https://github.com/PanJiaChen/vue-admin-template
12. # 桌面版
13. https://github.com/PanJiaChen/electron-vue-admin

3.6 图表展示

Highchars

1. https://www.highcharts.com.cn/

4. charset="utf-8">
5. name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
7. rel="stylesheet" href="/static/lib/layui/css/layui.css">
8. src="http://cdn.highcharts.com.cn/highcharts/highcharts.js">
9. {# src="/static/js/hichars.js">#}
11. class="layui-layout-body">
12. class="layui-layout layui-layout-admin">
13. class="layui-header">
14. class="layui-logo">layui 后台布局