Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
梦想橡皮擦
爬虫训练场
提交
2009a08b
爬
爬虫训练场
项目概览
梦想橡皮擦
/
爬虫训练场
通知
64
Star
7
Fork
1
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
爬
爬虫训练场
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
提交
2009a08b
编写于
1月 01, 2023
作者:
梦想橡皮擦
💬
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
AJAX案例
上级
53e60ab9
变更
12
隐藏空白更改
内联
并排
Showing
12 changed file
with
163 addition
and
19 deletion
+163
-19
app/__pycache__/routes.cpython-36.pyc
app/__pycache__/routes.cpython-36.pyc
+0
-0
app/antispider/__pycache__/index.cpython-36.pyc
app/antispider/__pycache__/index.cpython-36.pyc
+0
-0
app/file/__pycache__/index.cpython-36.pyc
app/file/__pycache__/index.cpython-36.pyc
+0
-0
app/file/index.py
app/file/index.py
+1
-1
app/model.py
app/model.py
+11
-0
app/school/__pycache__/index.cpython-36.pyc
app/school/__pycache__/index.cpython-36.pyc
+0
-0
app/school/index.py
app/school/index.py
+26
-3
app/static/mp4/demo.mp4
app/static/mp4/demo.mp4
+0
-0
app/templates/common/footer.html
app/templates/common/footer.html
+1
-1
app/templates/file/mp4.html
app/templates/file/mp4.html
+4
-5
app/templates/index.html
app/templates/index.html
+7
-9
app/templates/school/ajax_list.html
app/templates/school/ajax_list.html
+113
-0
未找到文件。
app/__pycache__/routes.cpython-36.pyc
浏览文件 @
2009a08b
无法预览此类型文件
app/antispider/__pycache__/index.cpython-36.pyc
浏览文件 @
2009a08b
无法预览此类型文件
app/file/__pycache__/index.cpython-36.pyc
浏览文件 @
2009a08b
无法预览此类型文件
app/file/index.py
浏览文件 @
2009a08b
...
...
@@ -11,7 +11,7 @@ def common_file():
@
f
.
route
(
'/mp4'
)
def
mp4_file
():
mp4_url
=
"http
://clips.vorwaerts-gmbh.de/big_buck_bunny
.mp4"
mp4_url
=
"http
s://img.tukuppt.com/video_show/2405179/00/01/53/5b45947d977b8
.mp4"
return
render_template
(
'file/mp4.html'
,
file
=
mp4_url
)
...
...
app/model.py
浏览文件 @
2009a08b
...
...
@@ -25,6 +25,17 @@ class School(db.Model, EntityBase):
category
=
db
.
Column
(
db
.
String
(
255
))
batchTimes
=
db
.
Column
(
db
.
String
(
255
))
def
to_dict
(
self
):
return
{
'name'
:
self
.
name
,
'province'
:
self
.
province
,
'city'
:
self
.
city
,
'feature'
:
self
.
feature
,
'hotValue'
:
self
.
hotValue
,
'pic'
:
self
.
pic
,
'category'
:
self
.
category
,
'batchTimes'
:
self
.
batchTimes
}
class
Csdn
(
db
.
Model
,
EntityBase
):
...
...
app/school/__pycache__/index.cpython-36.pyc
浏览文件 @
2009a08b
无法预览此类型文件
app/school/index.py
浏览文件 @
2009a08b
...
...
@@ -52,10 +52,9 @@ def Pagination(page, total):
return
page_obj
@
s
.
route
(
'/list'
)
def
list_school
():
def
pagination_object
(
page
=
1
):
# schools = School.query.all()
page
=
int
(
request
.
args
.
get
(
"page"
,
1
))
# 获取页码
query
=
School
.
query
total
=
query
.
count
()
# 获取数据总量
...
...
@@ -65,6 +64,30 @@ def list_school():
data_list
=
query
.
offset
(
pagination
[
"offset"
]).
limit
(
pagination
[
"page_size"
]).
all
()
else
:
data_list
=
[]
data_list
=
[
row
.
to_dict
()
for
row
in
data_list
]
pagination
[
"data_list"
]
=
data_list
return
pagination
@
s
.
route
(
'/list'
)
def
list_school
():
page
=
int
(
request
.
args
.
get
(
"page"
,
1
))
# 获取页码
pagination
=
pagination_object
(
page
)
return
render_template
(
'school/index.html'
,
pagination
=
pagination
)
@
s
.
route
(
'ajax_list'
)
def
ajax_list
():
page
=
1
# 初始化第一页数据
pagination
=
pagination_object
(
page
)
return
render_template
(
'school/ajax_list.html'
,
pagination
=
pagination
)
@
s
.
route
(
'api2'
)
def
ajax_list_school
():
page
=
int
(
request
.
args
.
get
(
"page"
,
1
))
pagination
=
pagination_object
(
page
)
return
jsonify
(
pagination
)
app/static/mp4/demo.mp4
0 → 100644
浏览文件 @
2009a08b
文件已添加
app/templates/common/footer.html
浏览文件 @
2009a08b
<div
class=
"container-fluid text-center text-muted "
>
<hr>
<p>
爬虫训练场 Copyright © 梦想橡皮擦 冀ICP备2022009308号-1
</p>
<p>
爬虫训练场 Copyright © 梦想橡皮擦 冀ICP备2022009308号-1
项目仓库地址:
<a
href=
"https://gitcode.net/hihell/spider_playground"
>
GitCode
</a>
</p>
</div>
\ No newline at end of file
app/templates/file/mp4.html
浏览文件 @
2009a08b
{% extends "base.html" %}
{% block content %}
<div
class=
"container text-center mt-3"
>
<div
class=
"container
-fluid
text-center mt-3"
>
<div
class=
"row"
>
<div
class=
"col-12"
>
<div
class=
"embed-responsive embed-responsive-16by9"
>
<video
class=
"embed-responsive-item"
src=
"{{file}}"
controls
></video>
</div>
<div
class=
"col"
>
<video
src=
"{{file}}"
controls
style=
"width:60%;"
></video>
</div>
</div>
</div>
...
...
app/templates/index.html
浏览文件 @
2009a08b
...
...
@@ -73,24 +73,23 @@
</div>
</div>
<div
class=
"col mt-2"
>
<div
class=
"card
border-secondary
rounded-5 shadow-sm"
style=
"min-height:268px;min-width:300px;"
>
<div
class=
"card
border-info
rounded-5 shadow-sm"
style=
"min-height:268px;min-width:300px;"
>
<div
class=
"card-header text-center"
>
<h4
class=
"card-title"
>
单页爬虫
</h4>
<h4
class=
"card-title"
>
AJAX 爬虫
</h4>
<div
class=
"bg-danger text-white rounded p-1"
style=
"transform: rotate(20deg); position:absolute;right:0;top:0.5rem;"
>
最新更新
</div>
</div>
<div
class=
"card-body"
>
<p
class=
"card-text"
>
目标数据呈现在单一页面中,使用最简单的爬虫库可以直接采集,一般用正则表达式即可完成数据提取
。
</p>
<p
class=
"card-text"
>
本案例涉及的数据采用异步AJAX返回,编写爬虫时需要注意抓取接口相关信息
。
</p>
<p
class=
"card-text text-left"
>
难度:⭐
</p>
<p
class=
"card-text"
>
案例:
<a
href=
"/general/news"
class=
"card-link text-success"
>
新闻页
</a>
<a
href=
"/general/imgs"
class=
"card-link text-success"
>
图片清单
</a>
<a
href=
"/general/table"
class=
"card-link text-success"
>
表格
</a>
<a
href=
"/ss/ajax_list"
class=
"card-link text-success"
>
AJAX学校数据
</a>
</p>
</div>
<div
class=
"card-footer text-end"
>
<a
href=
"#"
class=
"btn btn-primary card-link "
>
学习博客
</a>
</div>
</div>
</div>
...
...
@@ -141,7 +140,6 @@
</p>
</div>
<div
class=
"card-footer text-end"
>
<a
href=
"#"
class=
"btn btn-primary card-link "
>
学习博客
</a>
</div>
...
...
app/templates/school/ajax_list.html
0 → 100644
浏览文件 @
2009a08b
{% extends "base.html" %}
{% block script %}
<script
type=
"text/javascript"
src=
"https://ajax.aspnetcdn.com/ajax/jQuery/jquery-3.6.0.min.js"
></script>
<script
type=
"text/javascript"
>
function
get_data
(
page
){
$
.
ajax
({
type
:
"
get
"
,
url
:
"
/ss/api2
"
,
data
:
{
page
:
page
},
success
:
function
(
response
)
{
// ajax 请求成功
render_data
(
response
);
// 修改分页数据
$
(
'
.prev
'
).
attr
(
'
page
'
,
response
[
"
prev_page
"
]);
$
(
'
.next
'
).
attr
(
'
page
'
,
response
[
"
next_page
"
])
;
console
.
log
(
"
AJAX request succeeded!
"
);
},
error
:
function
(
error
)
{
console
.
log
(
"
AJAX request failed:
"
+
error
);
}
});
}
function
render_data
(
response
){
data_list
=
response
[
"
data_list
"
];
if
(
data_list
.
length
>
0
){
$
(
'
#school_list
'
).
empty
();
$
.
each
(
data_list
,
function
(
index
,
item
){
var
row
=
$
(
'
<div>
'
,
{
'
class
'
:
'
row mt-3
'
,
'
data-custom-attribute
'
:
'
value
'
});
var
col
=
$
(
'
<div>
'
,
{
'
class
'
:
'
col
'
});
var
d_flex
=
$
(
'
<div>
'
,
{
'
class
'
:
'
d-flex
'
});
d_flex
.
append
(
'
<div class="flex-shrink-0"><a href="#"><img class="rounded-pill img-thumbnail" width="64" height="64" src="
'
+
item
.
pic
+
'
" alt=""></a></div>
'
);
// 生成一下标签代码
var
badge
=
""
;
$
.
each
(
item
.
feature
.
split
(
'
,
'
),
function
(
i
,
f
){
badge
+=
'
<span class="badge rounded-pill bg-primary">
'
+
f
+
'
</span>
'
;
});
d_flex
.
append
(
'
<div class="flex-grow-1 ms-3"><h5 class="float-start pe-3">
'
+
item
.
name
+
'
</h5><p class="ms-3">
'
+
badge
+
'
</p><p><em>所在省市:<span class="text-black-50">
'
+
item
.
province
+
'
--
'
+
item
.
city
+
'
</span></em></p></div>
'
)
col
.
append
(
d_flex
);
row
.
append
(
col
);
$
(
'
#school_list
'
).
append
(
row
);
})
}
}
$
(
function
(){
$
(
'
.page-item
'
).
on
(
'
click
'
,
function
(){
page
=
$
(
this
).
attr
(
'
page
'
);
// 获取数据
get_data
(
page
);
})
})
</script>
{% endblock script %}
{% block content %}
<div
class=
"container"
id=
"school_list"
>
{% for school in pagination.data_list %}
<div
class=
"row mt-3"
>
<div
class=
"col"
>
<div
class=
"d-flex"
>
<div
class=
"flex-shrink-0"
>
<a
href=
"#"
>
<img
class=
"rounded-pill img-thumbnail"
width=
"64"
height=
"64"
src=
"{{school.pic}}"
alt=
""
>
</a>
</div>
<div
class=
"flex-grow-1 ms-3"
>
<h5
class=
"float-start pe-3"
>
{{school.name}}
</h5>
<p
class=
"ms-3"
>
{% for fea in school.feature.split(',') %}
<span
class=
"badge rounded-pill bg-primary"
>
{{fea}}
</span>
{% endfor %}
</p>
<p><em>
所在省市:
<span
class=
"text-black-50"
>
{{school.province}} -- {{school.city}}
</span></em></p>
</div>
</div>
</div>
</div>
{% endfor %}
</div>
<div
class=
"container"
>
<div
class=
"row"
>
<div
class=
"col"
>
<span
class=
"text-dark float-end align-middle"
style=
"line-height: 40px;"
>
合计 {{pagination.total}} 条数据
</span>
<ul
class=
"pagination float-end"
>
<li
class=
"page-item prev"
page=
"{{pagination.prev_page}}"
>
<a
class=
"page-link"
href=
"#"
>
上一页
</a>
</li>
<li
class=
"page-item next"
page=
"{{ pagination.next_page }}"
><a
class=
"page-link"
href=
"#"
>
下一页
</a>
</li>
</ul>
</div>
</div>
</div>
{% endblock %}
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录