Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
疯人忠
Cvat
提交
6fad1764
C
Cvat
项目概览
疯人忠
/
Cvat
通知
1
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
C
Cvat
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
未验证
提交
6fad1764
编写于
2月 28, 2022
作者:
A
Andrey Zhavoronkov
提交者:
GitHub
2月 28, 2022
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Added validation for URLs which used as remote data source (#4387)
上级
a3bc6ed2
变更
3
显示空白变更内容
内联
并排
Showing
3 changed file
with
45 addition
and
0 deletion
+45
-0
CHANGELOG.md
CHANGELOG.md
+1
-0
cvat/apps/engine/task.py
cvat/apps/engine/task.py
+43
-0
cvat/requirements/base.txt
cvat/requirements/base.txt
+1
-0
未找到文件。
CHANGELOG.md
浏览文件 @
6fad1764
...
...
@@ -81,6 +81,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Security
-
Updated ELK to 6.8.23 which uses log4j 2.17.1 (
<https://github.com/openvinotoolkit/cvat/pull/4206>
)
-
Added validation for URLs which used as remote data source (
<https://github.com/openvinotoolkit/cvat/pull/4387>
)
## \[1.7.0] - 2021-11-15
...
...
cvat/apps/engine/task.py
浏览文件 @
6fad1764
...
...
@@ -6,6 +6,7 @@
import
itertools
import
os
import
sys
from
rest_framework.serializers
import
ValidationError
import
rq
import
re
import
shutil
...
...
@@ -14,6 +15,8 @@ from traceback import print_exception
from
urllib
import
parse
as
urlparse
from
urllib
import
request
as
urlrequest
import
requests
import
ipaddress
import
dns.resolver
import
django_rq
from
django.conf
import
settings
...
...
@@ -203,6 +206,45 @@ def _validate_manifest(manifests, root_dir):
raise
Exception
(
'Invalid manifest was uploaded'
)
return
None
def
_validate_url
(
url
):
def
_validate_ip_address
(
ip_address
):
if
not
ip_address
.
is_global
:
raise
ValidationError
(
'Non public IP address
\'
{}
\'
is provided!'
.
format
(
ip_address
))
ALLOWED_SCHEMES
=
[
'http'
,
'https'
]
parsed_url
=
urlparse
.
urlparse
(
url
)
if
parsed_url
.
scheme
not
in
ALLOWED_SCHEMES
:
raise
ValueError
(
'Unsupported URL sheme: {}. Only http and https are supported'
.
format
(
parsed_url
.
scheme
))
try
:
ip_address
=
ipaddress
.
ip_address
(
parsed_url
.
hostname
)
_validate_ip_address
(
ip_address
)
except
ValueError
as
_
:
ip_v4_records
=
None
ip_v6_records
=
None
try
:
ip_v4_records
=
dns
.
resolver
.
query
(
parsed_url
.
hostname
,
'A'
)
for
record
in
ip_v4_records
:
_validate_ip_address
(
ipaddress
.
ip_address
(
record
.
to_text
()))
except
ValidationError
:
raise
except
Exception
as
e
:
slogger
.
glob
.
info
(
'Cannot get A record for domain
\'
{}
\'
: {}'
.
format
(
parsed_url
.
hostname
,
e
))
try
:
ip_v6_records
=
dns
.
resolver
.
query
(
parsed_url
.
hostname
,
'AAAA'
)
for
record
in
ip_v6_records
:
_validate_ip_address
(
ipaddress
.
ip_address
(
record
.
to_text
()))
except
ValidationError
:
raise
except
Exception
as
e
:
slogger
.
glob
.
info
(
'Cannot get AAAA record for domain
\'
{}
\'
: {}'
.
format
(
parsed_url
.
hostname
,
e
))
if
not
ip_v4_records
and
not
ip_v6_records
:
raise
ValidationError
(
'Cannot resolve IP address for domain
\'
{}
\'
'
.
format
(
parsed_url
.
hostname
))
def
_download_data
(
urls
,
upload_dir
):
job
=
rq
.
get_current_job
()
local_files
=
{}
...
...
@@ -210,6 +252,7 @@ def _download_data(urls, upload_dir):
name
=
os
.
path
.
basename
(
urlrequest
.
url2pathname
(
urlparse
.
urlparse
(
url
).
path
))
if
name
in
local_files
:
raise
Exception
(
"filename collision: {}"
.
format
(
name
))
_validate_url
(
url
)
slogger
.
glob
.
info
(
"Downloading: {}"
.
format
(
url
))
job
.
meta
[
'status'
]
=
'{} is being downloaded..'
.
format
(
url
)
job
.
save_meta
()
...
...
cvat/requirements/base.txt
浏览文件 @
6fad1764
...
...
@@ -52,3 +52,4 @@ datumaro==0.2.0 --no-binary=datumaro
urllib3>=1.26.5 # not directly required, pinned by Snyk to avoid a vulnerability
natsort==8.0.0
mistune>=2.0.1 # not directly required, pinned by Snyk to avoid a vulnerability
dnspython==2.2.0
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录