diff --git "a/Day36-40/\351\235\236\345\205\263\347\263\273\345\236\213\346\225\260\346\215\256\345\272\223Redis.md" "b/Day36-40/\351\235\236\345\205\263\347\263\273\345\236\213\346\225\260\346\215\256\345\272\223Redis.md" index 76cb580c5783bb9b8d1224a226e10cf1c6f8e234..9a0ae887458f36c15c3020d2f3f8f02293ef480e 100644 --- "a/Day36-40/\351\235\236\345\205\263\347\263\273\345\236\213\346\225\260\346\215\256\345\272\223Redis.md" +++ "b/Day36-40/\351\235\236\345\205\263\347\263\273\345\236\213\346\225\260\346\215\256\345\272\223Redis.md" @@ -6,7 +6,7 @@ ### Redis安装和配置 - + ### Redis的数据类型 diff --git "a/Day66-75/03.\345\255\230\345\202\250\346\225\260\346\215\256.md" "b/Day66-75/03.\345\255\230\345\202\250\346\225\260\346\215\256.md" index 31dd1fa821a11b493fab871e1cc5659b17997b37..6849a51085b3d18ba0b51864b574c77f9034ce53 100644 --- "a/Day66-75/03.\345\255\230\345\202\250\346\225\260\346\215\256.md" +++ "b/Day66-75/03.\345\255\230\345\202\250\346\225\260\346\215\256.md" @@ -6,9 +6,196 @@ ### 使用NoSQL -#### Redis +#### Redis简介 + +Redis是REmote DIctionary Server的缩写,它是一个用ANSI C编写的高性能的key-value存储系统,与其他的key-value存储系统相比,Redis有以下一些特点(也是优点): + +- Redis的读写性能极高,并且有丰富的特性(发布/订阅、事务、通知等)。 +- Redis支持数据的持久化(RDB和AOF两种方式),可以将内存中的数据保存在磁盘中,重启的时候可以再次加载进行使用。 +- Redis不仅仅支持简单的key-value类型的数据,同时还提供hash、list、set,zset、hyperloglog、geo等数据类型。 +- Redis支持主从复制(实现读写分析)以及哨兵模式(监控master是否宕机并调整配置)。 + +#### Redis的安装和配置 + +可以使用Linux系统的包管理工具(如yum)来安装Redis,也可以通过在Redis的[官方网站](https://redis.io/)下载Redis的[源代码](http://download.redis.io/releases/redis-3.2.11.tar.gz)解压缩解归档之后进行构件安装。 + +```Shell + +# wget http://download.redis.io/releases/redis-3.2.11.tar.gz +# gunzip redis-3.2.11.tar.gz +# tar -xvf redis-3.2.11.tar +# cd redis-3.2.11 +# make && make install +``` + +接下来我们将redis-3.2.11目录下的redis.conf配置文件复制到用户主目录下并修改配置文件(如果你对配置文件不是很有把握就不要直接修改而是先复制一份再修改这个副本)。 + +```Shell + +# cd .. +# cp redis-3.2.11/redis.conf redis.conf +# vim redis.conf +``` + +配置将Redis服务绑定到指定的IP地址和端口。 + +![](./res/redis-bind.png) + +![](./res/redis-port.png) + +配置底层有多少个数据库。 + +![](./res/redis-database.png) + +配置Redis的持久化机制 - RDB。 + +![](./res/redis-save.png) + +![](./res/redis-rdb.png) + +配置Redis的持久化机制 - AOF。 + +![](./res/redis-aof.png) + +配置访问Redis服务器的验证口令。 + +![](./res/redis-security.png) + +配置Redis的主从复制,通过主从复制可以实现读写分离。 + +![](./res/redis-replication.png) + +配置慢查询日志。 + +![](./res/redis-slow-log.png) + +这样我们就完成了Redis的基本配置,如果对上面的东西感到困惑,可以先系统的了解一下Redis,[《Redis开发与运维》](https://item.jd.com/12121730.html)是一本不错的入门读物,而[《Redis实战》](https://item.jd.com/11791607.html)是不错的进阶读物。 + +#### Redis的服务器和客户端 + +接下来启动Redis服务器,可以将服务器放在后台去运行。 + +```Shell + +# redis-server redis.conf & + + _.-``__ ''-._ + _.-`` `. `_. ''-._ Redis 3.2.11 (00000000/0) 64 bit + .-`` .-```. ```\/ _.,_ ''-._ + ( ' , .-` | `, ) Running in standalone mode + |`-._`-...-` __...-.``-._|'` _.-'| Port: 6379 + | `-._ `._ / _.-' | PID: 12345 + `-._ `-._ `-./ _.-' _.-' + |`-._`-._ `-.__.-' _.-'_.-'| + | `-._`-._ _.-'_.-' | http://redis.io + `-._ `-._`-.__.-'_.-' _.-' + |`-._`-._ `-.__.-' _.-'_.-'| + | `-._`-._ _.-'_.-' | + `-._ `-._`-.__.-'_.-' _.-' + `-._ `-.__.-' _.-' + `-._ _.-' + `-.__.-' +``` + +接下来,我们尝试用Redis客户端去连接服务器。 + +```Shell + +# redis-cli -h 172.18.61.250 -p 6379 +172.18.61.250:6379> auth 1qaz2wsx +OK +172.18.61.250:6379> ping +PONG +172.18.61.250:6379> +``` + +Redis有着非常丰富的数据类型,也有很多的命令来操作这些数据,具体的内容可以查看[Redis命令参考](http://redisdoc.com/),在这个网站上,除了Redis的命令参考,还有Redis的详细文档,其中包括了通知、事务、主从复制、持久化、哨兵、集群等内容。 + +```Shell + +172.18.61.250:6379> set username admin +OK +172.18.61.250:6379> get username +"admin" +172.18.61.250:6379> hset student1 name hao +(integer) 0 +172.18.61.250:6379> hset student1 age 38 +(integer) 1 +172.18.61.250:6379> hset student1 gender male +(integer) 1 +172.18.61.250:6379> hgetall student1 +1) "name" +2) "hao" +3) "age" +4) "38" +5) "gender" +6) "male" +172.18.61.250:6379> lpush num 1 2 3 4 5 +(integer) 5 +172.18.61.250:6379> lrange num 0 -1 +1) "5" +2) "4" +3) "3" +4) "2" +5) "1" +172.18.61.250:6379> sadd fruits apple banana orange apple grape grape +(integer) 4 +172.18.61.250:6379> scard fruits +(integer) 4 +172.18.61.250:6379> smembers fruits +1) "grape" +2) "orange" +3) "banana" +4) "apple" +172.18.61.250:6379> zadd scores 90 zhao 78 qian 66 sun 95 lee +(integer) 4 +172.18.61.250:6379> zrange scores 0 -1 +1) "sun" +2) "qian" +3) "zhao" +4) "lee" +172.18.61.250:6379> zrevrange scores 0 -1 +1) "lee" +2) "zhao" +3) "qian" +4) "sun" +``` + +#### 在Python程序中使用Redis + +可以使用pip安装redis模块。redis模块的核心是名为Redis的类,该类的对象代表一个Redis客户端,通过该客户端可以向Redis服务器发送命令并获取执行的结果。上面我们在Redis客户端中使用的命令基本上就是Redis对象可以接收的消息,所以如果了解了Redis的命令就可以在Python中玩转Redis。 + +```Shell + +$ pip3 install redis +$ python3 +``` + +```Python + +>>> import redis +>>> client = redis.Redis(host='1.2.3.4', port=6379, password='1qaz2wsx') +>>> client.set('username', 'admin') +True +>>> client.hset('student', 'name', 'hao') +1 +>>> client.hset('student', 'age', 38) +1 +>>> client.keys('*') +[b'username', b'student'] +>>> client.get('username') +b'admin' +>>> client.hgetall('student') +{b'name': b'hao', b'age': b'38'} +``` + +#### MongoDB简介 + +#### MongoDB的安装和配置 + +#### 使用MongoDB实现CRUD操作 + -#### Mongo diff --git a/Day66-75/code/example05.py b/Day66-75/code/example05.py index b54510aa50c45b6261ad534246b904f5d6ac8259..e48fa940e9dc803cff970b3084f4903036b51362 100644 --- a/Day66-75/code/example05.py +++ b/Day66-75/code/example05.py @@ -48,7 +48,7 @@ def get_matched_parts(page_html, pattern_str, pattern_ignore_case=re.I): return pattern_regex.findall(page_html) if page_html else [] -# 开始执行爬虫程序并对指定的数据进行持久化操作 +# 开始执行爬虫程序 def start_crawl(seed_url, match_pattern, *, max_depth=-1): client = redis.Redis(host='120.77.222.217', port=11223, password='1qaz2wsx') charsets = ('utf-8', 'gbk', 'gb2312') diff --git a/Day66-75/res/redis-aof.png b/Day66-75/res/redis-aof.png new file mode 100644 index 0000000000000000000000000000000000000000..c760bf4ad0d0fbb7154fc4de633ba73e1f01a8ef Binary files /dev/null and b/Day66-75/res/redis-aof.png differ diff --git a/Day66-75/res/redis-bind.png b/Day66-75/res/redis-bind.png new file mode 100644 index 0000000000000000000000000000000000000000..8bf32eda492484e34bf4457e6c1fc49f14fe44ec Binary files /dev/null and b/Day66-75/res/redis-bind.png differ diff --git a/Day66-75/res/redis-database.png b/Day66-75/res/redis-database.png new file mode 100644 index 0000000000000000000000000000000000000000..290d430ae2080707b5414abb573e48648727f43d Binary files /dev/null and b/Day66-75/res/redis-database.png differ diff --git a/Day66-75/res/redis-port.png b/Day66-75/res/redis-port.png new file mode 100644 index 0000000000000000000000000000000000000000..2b077bc1cb9f118427a7dfe1f3840fcc051690eb Binary files /dev/null and b/Day66-75/res/redis-port.png differ diff --git a/Day66-75/res/redis-rdb.png b/Day66-75/res/redis-rdb.png new file mode 100644 index 0000000000000000000000000000000000000000..8546e72acee4132fa418d0b7a84183a0aaaa1019 Binary files /dev/null and b/Day66-75/res/redis-rdb.png differ diff --git a/Day66-75/res/redis-replication.png b/Day66-75/res/redis-replication.png new file mode 100644 index 0000000000000000000000000000000000000000..9b68c963ec7bfccc35381afcba82ce786af11381 Binary files /dev/null and b/Day66-75/res/redis-replication.png differ diff --git a/Day66-75/res/redis-save.png b/Day66-75/res/redis-save.png new file mode 100644 index 0000000000000000000000000000000000000000..d721dd860ed696821f9a06696975eaba15b77cf9 Binary files /dev/null and b/Day66-75/res/redis-save.png differ diff --git a/Day66-75/res/redis-security.png b/Day66-75/res/redis-security.png new file mode 100644 index 0000000000000000000000000000000000000000..4eed293a9102a4f04bcfd7ce790960a73954ec95 Binary files /dev/null and b/Day66-75/res/redis-security.png differ diff --git a/Day66-75/res/redis-slow-log.png b/Day66-75/res/redis-slow-log.png new file mode 100644 index 0000000000000000000000000000000000000000..9af121fdab8423e37246b30a5e0e4146b61800af Binary files /dev/null and b/Day66-75/res/redis-slow-log.png differ