Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
ssssssss-team
spider-flow
提交
6543a74c
S
spider-flow
项目概览
ssssssss-team
/
spider-flow
10 个月 前同步成功
通知
0
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
S
spider-flow
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
前往新版Gitcode,体验更适合开发者的 AI 搜索 >>
提交
6543a74c
编写于
4月 11, 2020
作者:
N
nekolr
浏览文件
操作
浏览文件
下载
差异文件
Merge remote-tracking branch 'upstream/dev' into dev
上级
0d3fba67
ed4ec48d
变更
3
隐藏空白更改
内联
并排
Showing
3 changed file
with
75 addition
and
8 deletion
+75
-8
spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ElementsFunctionExtension.java
...xecutor/function/extension/ElementsFunctionExtension.java
+73
-5
spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ResponseFunctionExtension.java
...xecutor/function/extension/ResponseFunctionExtension.java
+1
-1
spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/SqlRowSetExtension.java
.../core/executor/function/extension/SqlRowSetExtension.java
+1
-2
未找到文件。
spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ElementsFunctionExtension.java
浏览文件 @
6543a74c
package
org.spiderflow.core.executor.function.extension
;
import
java.util.List
;
import
org.jsoup.nodes.Element
;
import
org.jsoup.select.Elements
;
import
org.spiderflow.annotation.Comment
;
import
org.spiderflow.annotation.Example
;
import
org.spiderflow.annotation.Return
;
import
org.spiderflow.core.utils.ExtractUtils
;
import
org.spiderflow.executor.FunctionExtension
;
import
org.springframework.stereotype.Component
;
import
java.util.ArrayList
;
import
java.util.List
;
@Component
public
class
ElementsFunctionExtension
implements
FunctionExtension
{
...
...
@@ -21,14 +21,12 @@ public class ElementsFunctionExtension implements FunctionExtension{
@Comment
(
"根据xpath提取内容"
)
@Example
(
"${elementsVar.xpath('//title/text()')}"
)
@Return
({
Element
.
class
,
String
.
class
})
public
static
String
xpath
(
Elements
elements
,
String
xpath
){
return
ExtractUtils
.
getValueByXPath
(
elements
,
xpath
);
}
@Comment
(
"根据xpath提取内容"
)
@Example
(
"${elementsVar.xpaths('//h2/text()')}"
)
@Return
({
Element
.
class
,
String
.
class
})
public
static
List
<
String
>
xpaths
(
Elements
elements
,
String
xpath
){
return
ExtractUtils
.
getValuesByXPath
(
elements
,
xpath
);
}
...
...
@@ -78,6 +76,76 @@ public class ElementsFunctionExtension implements FunctionExtension{
}
return
null
;
}
@Comment
(
"返回所有attr"
)
@Example
(
"${elementsVar.attrs('href')}"
)
public
static
List
<
String
>
attrs
(
Elements
elements
,
String
key
){
List
<
String
>
list
=
new
ArrayList
<>(
elements
.
size
());
for
(
Element
element
:
elements
)
{
list
.
add
(
element
.
attr
(
key
));
}
return
list
;
}
@Comment
(
"返回所有value"
)
@Example
(
"${elementsVar.vals()}"
)
public
static
List
<
String
>
vals
(
Elements
elements
){
List
<
String
>
list
=
new
ArrayList
<>(
elements
.
size
());
for
(
Element
element
:
elements
)
{
list
.
add
(
element
.
val
());
}
return
list
;
}
@Comment
(
"返回所有text"
)
@Example
(
"${elementsVar.texts()}"
)
public
static
List
<
String
>
texts
(
Elements
elements
){
List
<
String
>
list
=
new
ArrayList
<>(
elements
.
size
());
for
(
Element
element
:
elements
)
{
list
.
add
(
element
.
text
());
}
return
list
;
}
@Comment
(
"返回所有html"
)
@Example
(
"${elementsVar.htmls()}"
)
public
static
List
<
String
>
htmls
(
Elements
elements
){
List
<
String
>
list
=
new
ArrayList
<>(
elements
.
size
());
for
(
Element
element
:
elements
)
{
list
.
add
(
element
.
html
());
}
return
list
;
}
@Comment
(
"返回所有outerHtml"
)
@Example
(
"${elementsVar.outerHtmls()}"
)
public
static
List
<
String
>
outerHtmls
(
Elements
elements
){
List
<
String
>
list
=
new
ArrayList
<>(
elements
.
size
());
for
(
Element
element
:
elements
)
{
list
.
add
(
element
.
outerHtml
());
}
return
list
;
}
@Comment
(
"返回所有ownTexts"
)
@Example
(
"${elementsVar.ownTexts()}"
)
public
static
List
<
String
>
ownTexts
(
Elements
elements
){
List
<
String
>
list
=
new
ArrayList
<>(
elements
.
size
());
for
(
Element
element
:
elements
)
{
list
.
add
(
element
.
ownText
());
}
return
list
;
}
@Comment
(
"返回所有wholeText"
)
@Example
(
"${elementsVar.wholeTexts()}"
)
public
static
List
<
String
>
wholeTexts
(
Elements
elements
){
List
<
String
>
list
=
new
ArrayList
<>(
elements
.
size
());
for
(
Element
element
:
elements
)
{
list
.
add
(
element
.
wholeText
());
}
return
list
;
}
@Comment
(
"根据css选择器提取内容"
)
@Example
(
"${elementsVar.selectors('div > a')}"
)
...
...
spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ResponseFunctionExtension.java
浏览文件 @
6543a74c
...
...
@@ -27,7 +27,7 @@ public class ResponseFunctionExtension implements FunctionExtension {
@Comment
(
"将请求结果转为Element对象"
)
@Example
(
"${resp.element()}"
)
public
static
Element
element
(
SpiderResponse
response
)
{
return
Jsoup
.
parse
(
response
.
getHtml
());
return
Jsoup
.
parse
(
response
.
getHtml
()
,
response
.
getUrl
()
);
}
@Comment
(
"根据xpath在请求结果中查找"
)
...
...
spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/SqlRowSetExtension.java
浏览文件 @
6543a74c
...
...
@@ -21,8 +21,7 @@ public class SqlRowSetExtension implements FunctionExtension {
@Example
(
"${rs.nextToMap()}"
)
public
static
Map
<
String
,
Object
>
nextToMap
(
SqlRowSet
sqlRowSet
)
{
try
{
boolean
next
=
sqlRowSet
.
next
();
if
(!
next
)
{
if
(!
sqlRowSet
.
next
())
{
return
null
;
}
String
[]
columnNames
=
sqlRowSet
.
getMetaData
().
getColumnNames
();
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录