Skip to content

爬
爬虫训练场

梦想橡皮擦 / 爬虫训练场

通知 63
Star 7
Fork 1
爬
爬虫训练场

体验新版 GitCode，发现更多精彩内容 >>

提交 fcc494f9 编写于 作者: 梦想橡皮擦's avatar 梦想橡皮擦 💬
浏览文件
操作

第一个反爬案例 UA

上级 a87847a0
隐藏空白更改
内联 并排
Showing with 116 addition and 127 deletion
app/__init__.py
浏览文件 @ fcc494f9
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from .config import BaseConfig # 导入配置文件
from .config import BaseConfig # 导入配置文件
app = Flask(__name__)
app.config.from_object(BaseConfig) # 启用配置
app.config.from_object(BaseConfig) # 启用配置
db = SQLAlchemy()
db.init_app(app) # 初始化数据库
db.init_app(app) # 初始化数据库
# 导入路由配置
from app import routes
from app import general
from .school.index import *
from .file.index import *
from .antispider.index import *
app.register_blueprint(s)
app.register_blueprint(f)
app.register_blueprint(antispider)
app/antispider/index.py 0 → 100644
浏览文件 @ fcc494f9
from flask import Blueprint, jsonify, request, redirect, url_for
from flask import render_template
antispider = Blueprint('antispider', __name__, url_prefix='/as')
@antispider.before_request
def before_request():
user_agent = request.headers.get("User-Agent")
print(user_agent)
if "Mozilla" in user_agent:
# 允许请求通过
return None
else:
return "错误请求",403
@antispider.route('/show')
def index():
return render_template("antispider/ua_show.html")
@antispider.route('/error403')
def error403():
return "缺少关键参数", 403
app/routes.py
浏览文件 @ fcc494f9
......@@ -21,4 +21,9 @@ def bt():
@app.route('/mac')
def mac():
return render_template('_form.html')
\ No newline at end of file
return render_template('_form.html')
"""博客搭建清单"""
@app.route('/blog')
def blog():
return render_template('blog.html')
\ No newline at end of file
app/templates/antispider/ua_show.html 0 → 100644
浏览文件 @ fcc494f9
{% extends "base.html" %}
{% block content %}
<div class="container text-center pt-5" style="background:url({{url_for('static',filename='images/yy.jpg')}}) no-repeat top center">
<h3 class="" style="margin-top: 149px;">寓言故事：最可爱的孩子</h3>
<div style="min-height:440px;">
<p>有一个人在外地做事，托他的一位同乡带一件精巧又昂贵的玩具回家。</p>
<p>同乡问：“这东西带给谁呢?”</p>
<p>这个人认为自己的儿子长得伶俐聪明，是全村最可爱的孩子，就得意地说：“带给我们村里最可爱的孩子。”</p>
<p>同乡点点头，拿起东西走了。</p>
<p>过了几个月，这人回到家里，知道他的儿子并没有收到同乡带回来的玩具，便跑去问那个同乡：</p>
<p>“我托你带的玩具，怎么没有带给我的儿子?”</p>
<p>那同乡说：“你不是说带给全村最可爱的孩子吗?我认为我的儿子是全村最可爱的孩子，所以把玩具给了我的孩子啦!”</p>
</div>
</div>
{% endblock %}
app/templates/blog.html 0 → 100644
浏览文件 @ fcc494f9
{% extends "base.html" %}
{% block content %}
<div class="container pt-5">
<h3 class="text-success">【爬虫训练场】项目专栏教程清单</h3>
<hr>
<div class="row mt-3">
<div class="col">
<div class="list-group">
<a href="https://blog.csdn.net/hihell/article/details/128383663" target="_blank"
class="list-group-item list-group-item-action">01. 【永不断档】Python 爬虫训练场项目第一讲，环境初始化</a>
<a href="https://blog.csdn.net/hihell/article/details/128387250" target="_blank"
class="list-group-item list-group-item-action">02. 爬虫训练场项目，1小时掌握 Bootstrap 网格系统</a>
<a href="https://blog.csdn.net/hihell/article/details/128393999" target="_blank"
class="list-group-item list-group-item-action">03. 爬虫训练场项目前端基础，Bootstrap5排版、表格、图像</a>
<a href="https://blog.csdn.net/hihell/article/details/128396651" target="_blank"
class="list-group-item list-group-item-action">04. 爬虫训练场项目前端之 Bootstrap 信息提示框，按钮与按钮组，徽章，进度条</a>
<a href="https://blog.csdn.net/hihell/article/details/128397468" target="_blank"
class="list-group-item list-group-item-action">05. 爬虫训练场基础铺垫，BT加载器，分页，列表组，卡片，下拉菜单一文掌握</a>
<a href="https://blog.csdn.net/hihell/article/details/128399122" target="_blank"
class="list-group-item list-group-item-action">06. 爬虫训练场项目重要组件，导航栏+轮播图</a>
<a href="https://blog.csdn.net/hihell/article/details/128417565" target="_blank"
class="list-group-item list-group-item-action">07. 爬虫训练场项目，首页布局构建，完整版</a>
<a href="https://blog.csdn.net/hihell/article/details/128420246" target="_blank"
class="list-group-item list-group-item-action">08. 爬虫训练场，第一个爬虫目标页设计，单页爬虫案例</a>
<a href="https://blog.csdn.net/hihell/article/details/128429345" target="_blank"
class="list-group-item list-group-item-action">09. 爬虫训练场，分页爬虫案例设计Demo，打通 Python Flask 和 MySQL</a>
<a href="https://blog.csdn.net/hihell/article/details/128440699" target="_blank"
class="list-group-item list-group-item-action">10. 爬虫训练场，分页爬虫案例前端页面制作</a>
<a href="https://blog.csdn.net/hihell/article/details/128444515" target="_blank"
class="list-group-item list-group-item-action">11. 爬虫训练场，学校数据分页呈现，Bootstrap5 媒体组件</a>
<a href="https://dream.blog.csdn.net/article/details/128452409" target="_blank"
class="list-group-item list-group-item-action">12. 爬虫训练场项目，jinja2 模板继承，项目继续迭代</a>
<a href="https://blog.csdn.net/hihell/article/details/128465888?spm=1001.2014.3001.5501" target="_blank"
class="list-group-item list-group-item-action">13. 爬虫训练场集成文件采集案例，来学习一下怎么实现的</a>
<a href="https://blog.csdn.net/hihell/article/details/128473575?spm=1001.2014.3001.5501" target="_blank"
class="list-group-item list-group-item-action">14. UserAgent 反爬是如何实现的，来看看这篇博客 &</a>
</div>
</div>
</div>
</div>
{% endblock %}
app/templates/common/nav.html
浏览文件 @ fcc494f9
<nav class="navbar navbar-expand-sm bg-dark navbar-dark">
<div class="container-fluid">
<a href="/" class="navbar-brand">
<img src="/static/images/sucai.png" style="width:2rem;" class="rounded-pill me-2" alt=""/>爬虫训练场</a>
<img src="/static/images/0_hihell.png" style="width:2rem;" class="rounded-pill me-2" alt=""/>爬虫训练场</a>
<button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#collapsibleNavbar">
<span class="navbar-toggler-icon"></span>
......@@ -9,10 +9,10 @@
<div class="collapse navbar-collapse" id="collapsibleNavbar">
<ul class="navbar-nav">
<li class="nav-item">
<a href="#" class="nav-link">案例场</a>
<a href="/index" class="nav-link">案例场</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">爬虫博客</a>
<a class="nav-link" href="/blog">站点建设</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#">社群</a>
......
app/templates/index.html
浏览文件 @ fcc494f9
......@@ -97,130 +97,31 @@
</div>
<div class="container pt-5">
<h3>普通爬虫</h3>
<h3 class="text-danger">PC端反爬</h3>
<hr>
<div class="row align-items-stretch">
<div class="col-sm-6 col-md-3 mt-2">
<div class="card text-center border-secondary rounded-5 shadow-sm" style="height:268px;">
<div class="card-header">
<h4 class="card-title">单页爬虫</h4>
</div>
<div class="card-body">
<p class="card-text">单独一个网页数据展示，可直接采集，单独一个网页数据展示，可直接采集</p>
</div>
<div class="card-footer">
<a href="#" class="card-link">学习博客</a>
<a href="#" class="card-link">案例详情</a>
</div>
</div>
</div>
<div class="col-sm-6 col-md-3 mt-2">
<div class="card text-center border-secondary rounded-5 shadow-sm" style="height:268px;">
<div class="card-header">
<h4 class="card-title">单页爬虫</h4>
</div>
<div class="card-body">
<p class="card-text">单独一个网页数据展示，可直接采集，单独一个网页数据展示，可直接采集</p>
</div>
<div class="card-footer">
<a href="#" class="card-link">学习博客</a>
<a href="#" class="card-link">案例详情</a>
</div>
</div>
</div>
<div class="col-sm-6 col-md-3 mt-2">
<div class="card text-center border-secondary rounded-5 shadow-sm" style="height:268px;">
<div class="card-header">
<h4 class="card-title">单页爬虫</h4>
<div class="col mt-2">
<div class="card border-danger rounded-5 shadow-sm" style="min-height:268px;min-width:300px;">
<div class="card-header text-center">
<h4 class="card-title">UA 参数反爬</h4>
</div>
<div class="card-body">
<p class="card-text">单独一个网页数据展示，可直接采集，单独一个网页数据展示，可直接采集</p>
</div>
<div class="card-footer">
<a href="#" class="card-link">学习博客</a>
<a href="#" class="card-link">案例详情</a>
</div>
</div>
</div>
<div class="col-sm-6 col-md-3 mt-2">
<div class="card text-center border-secondary rounded-5 shadow-sm" style="height:268px;">
<div class="card-header">
<h4 class="card-title">单页爬虫</h4>
<p class="card-text">该案例要求爬虫程序构造请求头的时候，必须携带 User-Agent 参数，否则会返回403错误。</p>
<p class="card-text text-left">难度：⭐</p>
<p class="card-text">
案例：
<a href="/as/show" class="card-link text-success">预言故事</a>
</p>
</div>
<div class="card-body">
<p class="card-text">单独一个网页数据展示，可直接采集，单独一个网页数据展示，可直接采集</p>
<div class="card-footer text-end">
</div>
<div class="card-footer">
<a href="#" class="card-link">学习博客</a>
<a href="#" class="card-link">案例详情</a>
<a href="#" class="btn btn-primary card-link ">学习博客</a>
</div>
</div>
</div>
</div>
<div class="row align-items-stretch">
<div class="col-sm-6 col-md-3 mt-2">
<div class="card text-center border-secondary rounded-5 shadow-sm" style="height:268px;">
<div class="card-header">
<h4 class="card-title">单页爬虫</h4>
</div>
<div class="card-body">
<p class="card-text">单独一个网页数据展示，可直接采集，单独一个网页数据展示，可直接采集</p>
</div>
<div class="card-footer">
<a href="#" class="card-link">学习博客</a>
<a href="#" class="card-link">案例详情</a>
</div>
</div>
</div>
<div class="col-sm-6 col-md-3 mt-2">
<div class="card text-center border-secondary rounded-5 shadow-sm" style="height:268px;">
<div class="card-header">
<h4 class="card-title">单页爬虫</h4>
</div>
<div class="card-body">
<p class="card-text">单独一个网页数据展示，可直接采集，单独一个网页数据展示，可直接采集</p>
</div>
<div class="card-footer">
<a href="#" class="card-link">学习博客</a>
<a href="#" class="card-link">案例详情</a>
</div>
</div>
</div>
<div class="col-sm-6 col-md-3 mt-2">
<div class="card text-center border-secondary rounded-5 shadow-sm" style="height:268px;">
<div class="card-header">
<h4 class="card-title">单页爬虫</h4>
</div>
<div class="card-body">
<p class="card-text">单独一个网页数据展示，可直接采集，单独一个网页数据展示，可直接采集</p>
</div>
<div class="card-footer">
<a href="#" class="card-link">学习博客</a>
<a href="#" class="card-link">案例详情</a>
</div>
</div>
</div>
</div>
</div>
{% endblock %}
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或