Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
40749ddc
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
40749ddc
编写于
2月 18, 2016
作者:
G
Greg Hogan
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[FLINK-3277] [gelly] Use Value types in Gelly API
This closes #1671
上级
10898a90
变更
18
隐藏空白更改
内联
并排
Showing
18 changed file
with
138 addition
and
130 deletion
+138
-130
docs/apis/batch/libs/gelly.md
docs/apis/batch/libs/gelly.md
+12
-12
flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/examples/GraphMetrics.java
...in/java/org/apache/flink/graph/examples/GraphMetrics.java
+8
-7
flink-libraries/flink-gelly-examples/src/main/scala/org/apache/flink/graph/scala/examples/GraphMetrics.scala
.../org/apache/flink/graph/scala/examples/GraphMetrics.scala
+1
-1
flink-libraries/flink-gelly-scala/src/main/scala/org/apache/flink/graph/scala/Graph.scala
...a/src/main/scala/org/apache/flink/graph/scala/Graph.scala
+10
-12
flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/Graph.java
...ink-gelly/src/main/java/org/apache/flink/graph/Graph.java
+13
-6
flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/library/GSAPageRank.java
...main/java/org/apache/flink/graph/library/GSAPageRank.java
+5
-5
flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/library/PageRank.java
...rc/main/java/org/apache/flink/graph/library/PageRank.java
+5
-5
flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/spargel/ScatterGatherIteration.java
...rg/apache/flink/graph/spargel/ScatterGatherIteration.java
+35
-35
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/generator/CompleteGraphTest.java
...a/org/apache/flink/graph/generator/CompleteGraphTest.java
+4
-4
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/generator/CycleGraphTest.java
...java/org/apache/flink/graph/generator/CycleGraphTest.java
+4
-4
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/generator/EmptyGraphTest.java
...java/org/apache/flink/graph/generator/EmptyGraphTest.java
+2
-2
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/generator/GridGraphTest.java
.../java/org/apache/flink/graph/generator/GridGraphTest.java
+4
-4
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/generator/HypercubeGraphTest.java
.../org/apache/flink/graph/generator/HypercubeGraphTest.java
+4
-4
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/generator/PathGraphTest.java
.../java/org/apache/flink/graph/generator/PathGraphTest.java
+4
-4
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/generator/SingletonEdgeGraphTest.java
.../apache/flink/graph/generator/SingletonEdgeGraphTest.java
+4
-4
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/generator/StarGraphTest.java
.../java/org/apache/flink/graph/generator/StarGraphTest.java
+4
-4
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/DegreesITCase.java
...org/apache/flink/graph/test/operations/DegreesITCase.java
+13
-12
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/DegreesWithExceptionITCase.java
...ink/graph/test/operations/DegreesWithExceptionITCase.java
+6
-5
未找到文件。
docs/apis/batch/libs/gelly.md
浏览文件 @
40749ddc
...
...
@@ -346,13 +346,13 @@ DataSet<K> getVertexIds()
DataSet
<Tuple2
<
K
,
K
>
> getEdgeIds()
// get a DataSet of
<vertex
ID
,
in-degree
>
pairs for all vertices
DataSet
<Tuple2
<
K
,
Long
>
> inDegrees()
DataSet
<Tuple2
<
K
,
Long
Value
>
> inDegrees()
// get a DataSet of
<vertex
ID
,
out-degree
>
pairs for all vertices
DataSet
<Tuple2
<
K
,
Long
>
> outDegrees()
DataSet
<Tuple2
<
K
,
Long
Value
>
> outDegrees()
// get a DataSet of
<vertex
ID
,
degree
>
pairs for all vertices, where degree is the sum of in- and out- degrees
DataSet
<Tuple2
<
K
,
Long
>
> getDegrees()
DataSet
<Tuple2
<
K
,
Long
Value
>
> getDegrees()
// get the number of vertices
long numberOfVertices()
...
...
@@ -381,13 +381,13 @@ getVertexIds: DataSet[K]
getEdgeIds: DataSet[(K, K)]
// get a DataSet of
<vertex
ID
,
in-degree
>
pairs for all vertices
inDegrees: DataSet[(K, Long)]
inDegrees: DataSet[(K, Long
Value
)]
// get a DataSet of
<vertex
ID
,
out-degree
>
pairs for all vertices
outDegrees: DataSet[(K, Long)]
outDegrees: DataSet[(K, Long
Value
)]
// get a DataSet of
<vertex
ID
,
degree
>
pairs for all vertices, where degree is the sum of in- and out- degrees
getDegrees: DataSet[(K, Long)]
getDegrees: DataSet[(K, Long
Value
)]
// get the number of vertices
numberOfVertices: Long
...
...
@@ -519,13 +519,13 @@ Note that if the input dataset contains a key multiple times, all Gelly join met
{% highlight java %}
Graph
<Long
,
Double
,
Double
>
network = ...
DataSet
<Tuple2
<
Long
,
Long
>
> vertexOutDegrees = network.outDegrees();
DataSet
<Tuple2
<
Long
,
Long
Value
>
> vertexOutDegrees = network.outDegrees();
// assign the transition probabilities as the edge weights
Graph
<Long
,
Double
,
Double
>
networkWithWeights = network.joinWithEdgesOnSource(vertexOutDegrees,
new VertexJoinFunction
<Double
,
Long
>
() {
public Double vertexJoin(Double vertexValue, Long inputValue) {
return vertexValue / inputValue;
new VertexJoinFunction
<Double
,
Long
Value
>
() {
public Double vertexJoin(Double vertexValue, Long
Value
inputValue) {
return vertexValue / inputValue
.getValue()
;
}
});
{% endhighlight %}
...
...
@@ -535,10 +535,10 @@ Graph<Long, Double, Double> networkWithWeights = network.joinWithEdgesOnSource(v
{% highlight scala %}
val network: Graph[Long, Double, Double] = ...
val vertexOutDegrees: DataSet[(Long, Long)] = network.outDegrees
val vertexOutDegrees: DataSet[(Long, Long
Value
)] = network.outDegrees
// assign the transition probabilities as the edge weights
val networkWithWeights = network.joinWithEdgesOnSource(vertexOutDegrees, (v1: Double, v2: Long
) => v1 / v2
)
val networkWithWeights = network.joinWithEdgesOnSource(vertexOutDegrees, (v1: Double, v2: Long
Value) => v1 / v2.getValue
)
{% endhighlight %}
</div>
</div>
...
...
flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/examples/GraphMetrics.java
浏览文件 @
40749ddc
...
...
@@ -18,7 +18,6 @@
package
org.apache.flink.graph.examples
;
import
org.apache.flink.graph.examples.utils.ExampleUtils
;
import
org.apache.flink.api.common.ProgramDescription
;
import
org.apache.flink.api.common.functions.MapFunction
;
import
org.apache.flink.api.java.DataSet
;
...
...
@@ -27,6 +26,8 @@ import org.apache.flink.api.java.aggregation.Aggregations;
import
org.apache.flink.api.java.tuple.Tuple2
;
import
org.apache.flink.graph.Edge
;
import
org.apache.flink.graph.Graph
;
import
org.apache.flink.graph.examples.utils.ExampleUtils
;
import
org.apache.flink.types.LongValue
;
import
org.apache.flink.types.NullValue
;
/**
...
...
@@ -66,7 +67,7 @@ public class GraphMetrics implements ProgramDescription {
long
numEdges
=
graph
.
numberOfEdges
();
/** compute the average node degree **/
DataSet
<
Tuple2
<
Long
,
Long
>>
verticesWithDegrees
=
graph
.
getDegrees
();
DataSet
<
Tuple2
<
Long
,
Long
Value
>>
verticesWithDegrees
=
graph
.
getDegrees
();
DataSet
<
Double
>
avgNodeDegree
=
verticesWithDegrees
.
aggregate
(
Aggregations
.
SUM
,
1
).
map
(
new
AvgNodeDegreeMapper
(
numVertices
));
...
...
@@ -96,7 +97,7 @@ public class GraphMetrics implements ProgramDescription {
}
@SuppressWarnings
(
"serial"
)
private
static
final
class
AvgNodeDegreeMapper
implements
MapFunction
<
Tuple2
<
Long
,
Long
>,
Double
>
{
private
static
final
class
AvgNodeDegreeMapper
implements
MapFunction
<
Tuple2
<
Long
,
Long
Value
>,
Double
>
{
private
long
numberOfVertices
;
...
...
@@ -104,14 +105,14 @@ public class GraphMetrics implements ProgramDescription {
this
.
numberOfVertices
=
numberOfVertices
;
}
public
Double
map
(
Tuple2
<
Long
,
Long
>
sumTuple
)
{
return
(
double
)
(
sumTuple
.
f1
/
numberOfVertices
)
;
public
Double
map
(
Tuple2
<
Long
,
Long
Value
>
sumTuple
)
{
return
(
double
)
(
sumTuple
.
f1
.
getValue
()
/
numberOfVertices
)
;
}
}
@SuppressWarnings
(
"serial"
)
private
static
final
class
ProjectVertexId
implements
MapFunction
<
Tuple2
<
Long
,
Long
>,
Long
>
{
public
Long
map
(
Tuple2
<
Long
,
Long
>
value
)
{
return
value
.
f0
;
}
private
static
final
class
ProjectVertexId
implements
MapFunction
<
Tuple2
<
Long
,
LongValue
>,
Long
>
{
public
Long
map
(
Tuple2
<
Long
,
Long
Value
>
value
)
{
return
value
.
f0
;
}
}
@Override
...
...
flink-libraries/flink-gelly-examples/src/main/scala/org/apache/flink/graph/scala/examples/GraphMetrics.scala
浏览文件 @
40749ddc
...
...
@@ -61,7 +61,7 @@ object GraphMetrics {
/** compute the average node degree **/
val
verticesWithDegrees
=
graph
.
getDegrees
val
avgDegree
=
verticesWithDegrees
.
sum
(
1
).
map
(
in
=>
(
in
.
_2
/
numVertices
).
toDouble
)
val
avgDegree
=
verticesWithDegrees
.
sum
(
1
).
map
(
in
=>
(
in
.
_2
.
getValue
/
numVertices
).
toDouble
)
/** find the vertex with the maximum in-degree **/
val
maxInDegreeVertex
=
graph
.
inDegrees
.
max
(
1
).
map
(
in
=>
in
.
_1
)
...
...
flink-libraries/flink-gelly-scala/src/main/scala/org/apache/flink/graph/scala/Graph.scala
浏览文件 @
40749ddc
...
...
@@ -18,24 +18,22 @@
package
org.apache.flink.graph.scala
import
org.apache.flink.util.Preconditions
import
org.apache.flink.api.common.functions.
{
FilterFunction
,
MapFunction
}
import
org.apache.flink.api.common.typeinfo.TypeInformation
import
org.apache.flink.api.java.
{
tuple
=>
jtuple
}
import
org.apache.flink.api.scala._
import
org.apache.flink.graph._
import
org.apache.flink.graph.asm.translate.TranslateFunction
import
org.apache.flink.graph.validation.GraphValidator
import
org.apache.flink.graph.gsa.
{
ApplyFunction
,
GSAConfiguration
,
GatherFunction
,
SumFunction
}
import
org.apache.flink.graph.pregel.
{
ComputeFunction
,
MessageCombiner
,
VertexCentricConfiguration
}
import
org.apache.flink.graph.spargel.
{
MessagingFunction
,
ScatterGatherConfiguration
,
VertexUpdateFunction
}
import
org.apache.flink.graph.validation.GraphValidator
import
org.apache.flink.types.
{
LongValue
,
NullValue
}
import
org.apache.flink.util.Preconditions
import
org.apache.flink.
{
graph
=>
jg
}
import
_root_.scala.collection.JavaConverters._
import
_root_.scala.reflect.ClassTag
import
org.apache.flink.types.NullValue
import
org.apache.flink.graph.pregel.ComputeFunction
import
org.apache.flink.graph.pregel.MessageCombiner
import
org.apache.flink.graph.pregel.VertexCentricConfiguration
object
Graph
{
...
...
@@ -803,8 +801,8 @@ TypeInformation : ClassTag](jgraph: jg.Graph[K, VV, EV]) {
*
* @return A DataSet of Tuple2<vertexId, inDegree>
*/
def
inDegrees
()
:
DataSet
[(
K
,
Long
)]
=
{
wrap
(
jgraph
.
inDegrees
).
map
(
javatuple
=>
(
javatuple
.
f0
,
javatuple
.
f1
.
longValue
()
))
def
inDegrees
()
:
DataSet
[(
K
,
Long
Value
)]
=
{
wrap
(
jgraph
.
inDegrees
).
map
(
javatuple
=>
(
javatuple
.
f0
,
javatuple
.
f1
))
}
/**
...
...
@@ -812,8 +810,8 @@ TypeInformation : ClassTag](jgraph: jg.Graph[K, VV, EV]) {
*
* @return A DataSet of Tuple2<vertexId, outDegree>
*/
def
outDegrees
()
:
DataSet
[(
K
,
Long
)]
=
{
wrap
(
jgraph
.
outDegrees
).
map
(
javatuple
=>
(
javatuple
.
f0
,
javatuple
.
f1
.
longValue
()
))
def
outDegrees
()
:
DataSet
[(
K
,
Long
Value
)]
=
{
wrap
(
jgraph
.
outDegrees
).
map
(
javatuple
=>
(
javatuple
.
f0
,
javatuple
.
f1
))
}
/**
...
...
@@ -821,8 +819,8 @@ TypeInformation : ClassTag](jgraph: jg.Graph[K, VV, EV]) {
*
* @return A DataSet of Tuple2<vertexId, degree>
*/
def
getDegrees
()
:
DataSet
[(
K
,
Long
)]
=
{
wrap
(
jgraph
.
getDegrees
).
map
(
javatuple
=>
(
javatuple
.
f0
,
javatuple
.
f1
.
longValue
()
))
def
getDegrees
()
:
DataSet
[(
K
,
Long
Value
)]
=
{
wrap
(
jgraph
.
getDegrees
).
map
(
javatuple
=>
(
javatuple
.
f0
,
javatuple
.
f1
))
}
/**
...
...
flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/Graph.java
浏览文件 @
40749ddc
...
...
@@ -61,6 +61,7 @@ import org.apache.flink.graph.utils.Tuple2ToVertexMap;
import
org.apache.flink.graph.utils.Tuple3ToEdgeMap
;
import
org.apache.flink.graph.utils.VertexToTuple2Map
;
import
org.apache.flink.graph.validation.GraphValidator
;
import
org.apache.flink.types.LongValue
;
import
org.apache.flink.types.NullValue
;
import
org.apache.flink.util.Collector
;
...
...
@@ -867,25 +868,31 @@ public class Graph<K, VV, EV> {
*
* @return A DataSet of {@code Tuple2<vertexId, outDegree>}
*/
public
DataSet
<
Tuple2
<
K
,
Long
>>
outDegrees
()
{
public
DataSet
<
Tuple2
<
K
,
Long
Value
>>
outDegrees
()
{
return
vertices
.
coGroup
(
edges
).
where
(
0
).
equalTo
(
0
).
with
(
new
CountNeighborsCoGroup
<
K
,
VV
,
EV
>());
}
private
static
final
class
CountNeighborsCoGroup
<
K
,
VV
,
EV
>
implements
CoGroupFunction
<
Vertex
<
K
,
VV
>,
Edge
<
K
,
EV
>,
Tuple2
<
K
,
Long
>>
{
implements
CoGroupFunction
<
Vertex
<
K
,
VV
>,
Edge
<
K
,
EV
>,
Tuple2
<
K
,
LongValue
>>
{
private
LongValue
degree
=
new
LongValue
();
private
Tuple2
<
K
,
LongValue
>
vertexDegree
=
new
Tuple2
<>(
null
,
degree
);
@SuppressWarnings
(
"unused"
)
public
void
coGroup
(
Iterable
<
Vertex
<
K
,
VV
>>
vertex
,
Iterable
<
Edge
<
K
,
EV
>>
outEdges
,
Collector
<
Tuple2
<
K
,
Long
>>
out
)
{
Collector
<
Tuple2
<
K
,
Long
Value
>>
out
)
{
long
count
=
0
;
for
(
Edge
<
K
,
EV
>
edge
:
outEdges
)
{
count
++;
}
degree
.
setValue
(
count
);
Iterator
<
Vertex
<
K
,
VV
>>
vertexIterator
=
vertex
.
iterator
();
if
(
vertexIterator
.
hasNext
())
{
out
.
collect
(
new
Tuple2
<
K
,
Long
>(
vertexIterator
.
next
().
f0
,
count
));
vertexDegree
.
f0
=
vertexIterator
.
next
().
f0
;
out
.
collect
(
vertexDegree
);
}
else
{
throw
new
NoSuchElementException
(
"The edge src/trg id could not be found within the vertexIds"
);
}
...
...
@@ -897,7 +904,7 @@ public class Graph<K, VV, EV> {
*
* @return A DataSet of {@code Tuple2<vertexId, inDegree>}
*/
public
DataSet
<
Tuple2
<
K
,
Long
>>
inDegrees
()
{
public
DataSet
<
Tuple2
<
K
,
Long
Value
>>
inDegrees
()
{
return
vertices
.
coGroup
(
edges
).
where
(
0
).
equalTo
(
1
).
with
(
new
CountNeighborsCoGroup
<
K
,
VV
,
EV
>());
}
...
...
@@ -907,7 +914,7 @@ public class Graph<K, VV, EV> {
*
* @return A DataSet of {@code Tuple2<vertexId, degree>}
*/
public
DataSet
<
Tuple2
<
K
,
Long
>>
getDegrees
()
{
public
DataSet
<
Tuple2
<
K
,
Long
Value
>>
getDegrees
()
{
return
outDegrees
().
union
(
inDegrees
()).
groupBy
(
0
).
sum
(
1
);
}
...
...
flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/library/GSAPageRank.java
浏览文件 @
40749ddc
...
...
@@ -29,6 +29,7 @@ import org.apache.flink.graph.gsa.GSAConfiguration;
import
org.apache.flink.graph.gsa.GatherFunction
;
import
org.apache.flink.graph.gsa.Neighbor
;
import
org.apache.flink.graph.gsa.SumFunction
;
import
org.apache.flink.types.LongValue
;
/**
* This is an implementation of a simple PageRank algorithm, using a gather-sum-apply iteration.
...
...
@@ -56,8 +57,7 @@ public class GSAPageRank<K> implements GraphAlgorithm<K, Double, Double, DataSet
@Override
public
DataSet
<
Vertex
<
K
,
Double
>>
run
(
Graph
<
K
,
Double
,
Double
>
network
)
throws
Exception
{
DataSet
<
Tuple2
<
K
,
Long
>>
vertexOutDegrees
=
network
.
outDegrees
();
DataSet
<
Tuple2
<
K
,
LongValue
>>
vertexOutDegrees
=
network
.
outDegrees
();
Graph
<
K
,
Double
,
Double
>
networkWithWeights
=
network
.
joinWithEdgesOnSource
(
vertexOutDegrees
,
new
InitWeights
());
...
...
@@ -114,10 +114,10 @@ public class GSAPageRank<K> implements GraphAlgorithm<K, Double, Double, DataSet
}
@SuppressWarnings
(
"serial"
)
private
static
final
class
InitWeights
implements
EdgeJoinFunction
<
Double
,
Long
>
{
private
static
final
class
InitWeights
implements
EdgeJoinFunction
<
Double
,
Long
Value
>
{
public
Double
edgeJoin
(
Double
edgeValue
,
Long
inputValue
)
{
return
edgeValue
/
(
double
)
inputValue
;
public
Double
edgeJoin
(
Double
edgeValue
,
Long
Value
inputValue
)
{
return
edgeValue
/
(
double
)
inputValue
.
getValue
()
;
}
}
}
flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/library/PageRank.java
浏览文件 @
40749ddc
...
...
@@ -29,6 +29,7 @@ import org.apache.flink.graph.spargel.MessageIterator;
import
org.apache.flink.graph.spargel.MessagingFunction
;
import
org.apache.flink.graph.spargel.ScatterGatherConfiguration
;
import
org.apache.flink.graph.spargel.VertexUpdateFunction
;
import
org.apache.flink.types.LongValue
;
/**
* This is an implementation of a simple PageRank algorithm, using a scatter-gather iteration.
...
...
@@ -56,8 +57,7 @@ public class PageRank<K> implements GraphAlgorithm<K, Double, Double, DataSet<Ve
@Override
public
DataSet
<
Vertex
<
K
,
Double
>>
run
(
Graph
<
K
,
Double
,
Double
>
network
)
throws
Exception
{
DataSet
<
Tuple2
<
K
,
Long
>>
vertexOutDegrees
=
network
.
outDegrees
();
DataSet
<
Tuple2
<
K
,
LongValue
>>
vertexOutDegrees
=
network
.
outDegrees
();
Graph
<
K
,
Double
,
Double
>
networkWithWeights
=
network
.
joinWithEdgesOnSource
(
vertexOutDegrees
,
new
InitWeights
());
...
...
@@ -118,10 +118,10 @@ public class PageRank<K> implements GraphAlgorithm<K, Double, Double, DataSet<Ve
}
@SuppressWarnings
(
"serial"
)
private
static
final
class
InitWeights
implements
EdgeJoinFunction
<
Double
,
Long
>
{
private
static
final
class
InitWeights
implements
EdgeJoinFunction
<
Double
,
Long
Value
>
{
public
Double
edgeJoin
(
Double
edgeValue
,
Long
inputValue
)
{
return
edgeValue
/
(
double
)
inputValue
;
public
Double
edgeJoin
(
Double
edgeValue
,
Long
Value
inputValue
)
{
return
edgeValue
/
(
double
)
inputValue
.
getValue
()
;
}
}
...
...
flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/spargel/ScatterGatherIteration.java
浏览文件 @
40749ddc
...
...
@@ -309,28 +309,28 @@ public class ScatterGatherIteration<K, VV, Message, EV>
}
@SuppressWarnings
(
"serial"
)
private
static
final
class
VertexUpdateUdfVVWithDegrees
<
K
,
VV
,
Message
>
extends
VertexUpdateUdf
<
K
,
Tuple3
<
VV
,
Long
,
Long
>,
Message
>
{
private
static
final
class
VertexUpdateUdfVVWithDegrees
<
K
,
VV
,
Message
>
extends
VertexUpdateUdf
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>,
Message
>
{
private
VertexUpdateUdfVVWithDegrees
(
VertexUpdateFunction
<
K
,
Tuple3
<
VV
,
Long
,
Long
>,
Message
>
vertexUpdateFunction
,
TypeInformation
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>>
resultType
)
{
private
VertexUpdateUdfVVWithDegrees
(
VertexUpdateFunction
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>,
Message
>
vertexUpdateFunction
,
TypeInformation
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>>
resultType
)
{
super
(
vertexUpdateFunction
,
resultType
);
}
@Override
public
void
coGroup
(
Iterable
<
Tuple2
<
K
,
Message
>>
messages
,
Iterable
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>>
vertex
,
Collector
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>>
out
)
throws
Exception
{
public
void
coGroup
(
Iterable
<
Tuple2
<
K
,
Message
>>
messages
,
Iterable
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>>
vertex
,
Collector
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>>
out
)
throws
Exception
{
final
Iterator
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>>
vertexIter
=
vertex
.
iterator
();
final
Iterator
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>>
vertexIter
=
vertex
.
iterator
();
if
(
vertexIter
.
hasNext
())
{
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>
vertexWithDegrees
=
vertexIter
.
next
();
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>
vertexWithDegrees
=
vertexIter
.
next
();
@SuppressWarnings
(
"unchecked"
)
Iterator
<
Tuple2
<?,
Message
>>
downcastIter
=
(
Iterator
<
Tuple2
<?,
Message
>>)
(
Iterator
<?>)
messages
.
iterator
();
messageIter
.
setSource
(
downcastIter
);
vertexUpdateFunction
.
setInDegree
(
vertexWithDegrees
.
f1
.
f1
);
vertexUpdateFunction
.
setOutDegree
(
vertexWithDegrees
.
f1
.
f2
);
vertexUpdateFunction
.
setInDegree
(
vertexWithDegrees
.
f1
.
f1
.
getValue
()
);
vertexUpdateFunction
.
setOutDegree
(
vertexWithDegrees
.
f1
.
f2
.
getValue
()
);
vertexUpdateFunction
.
setOutputWithDegrees
(
vertexWithDegrees
,
out
);
vertexUpdateFunction
.
updateVertexFromScatterGatherIteration
(
vertexWithDegrees
,
messageIter
);
...
...
@@ -420,7 +420,7 @@ public class ScatterGatherIteration<K, VV, Message, EV>
@SuppressWarnings
(
"serial"
)
private
static
final
class
MessagingUdfWithEVsVVWithDegrees
<
K
,
VV
,
Message
,
EV
>
extends
MessagingUdfWithEdgeValues
<
K
,
Tuple3
<
VV
,
Long
,
Long
>,
VV
,
Message
,
EV
>
{
extends
MessagingUdfWithEdgeValues
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>,
VV
,
Message
,
EV
>
{
private
Vertex
<
K
,
VV
>
nextVertex
=
new
Vertex
<
K
,
VV
>();
...
...
@@ -430,19 +430,19 @@ public class ScatterGatherIteration<K, VV, Message, EV>
}
@Override
public
void
coGroup
(
Iterable
<
Edge
<
K
,
EV
>>
edges
,
Iterable
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>>
state
,
public
void
coGroup
(
Iterable
<
Edge
<
K
,
EV
>>
edges
,
Iterable
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>>
state
,
Collector
<
Tuple2
<
K
,
Message
>>
out
)
throws
Exception
{
final
Iterator
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>>
stateIter
=
state
.
iterator
();
final
Iterator
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>>
stateIter
=
state
.
iterator
();
if
(
stateIter
.
hasNext
())
{
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>
vertexWithDegrees
=
stateIter
.
next
();
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>
vertexWithDegrees
=
stateIter
.
next
();
nextVertex
.
setField
(
vertexWithDegrees
.
f0
,
0
);
nextVertex
.
setField
(
vertexWithDegrees
.
f1
.
f0
,
1
);
messagingFunction
.
setInDegree
(
vertexWithDegrees
.
f1
.
f1
);
messagingFunction
.
setOutDegree
(
vertexWithDegrees
.
f1
.
f2
);
messagingFunction
.
setInDegree
(
vertexWithDegrees
.
f1
.
f1
.
getValue
()
);
messagingFunction
.
setOutDegree
(
vertexWithDegrees
.
f1
.
f2
.
getValue
()
);
messagingFunction
.
set
((
Iterator
<?>)
edges
.
iterator
(),
out
,
vertexWithDegrees
.
getId
());
messagingFunction
.
sendMessages
(
nextVertex
);
...
...
@@ -505,13 +505,13 @@ public class ScatterGatherIteration<K, VV, Message, EV>
* @return the messaging function
*/
private
CoGroupOperator
<?,
?,
Tuple2
<
K
,
Message
>>
buildMessagingFunctionVerticesWithDegrees
(
DeltaIteration
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>,
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>>
iteration
,
DeltaIteration
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>,
Vertex
<
K
,
Tuple3
<
VV
,
LongValue
,
LongValue
>>>
iteration
,
TypeInformation
<
Tuple2
<
K
,
Message
>>
messageTypeInfo
,
int
whereArg
,
int
equalToArg
,
DataSet
<
LongValue
>
numberOfVertices
)
{
// build the messaging function (co group)
CoGroupOperator
<?,
?,
Tuple2
<
K
,
Message
>>
messages
;
MessagingUdfWithEdgeValues
<
K
,
Tuple3
<
VV
,
Long
,
Long
>,
VV
,
Message
,
EV
>
messenger
=
MessagingUdfWithEdgeValues
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>,
VV
,
Message
,
EV
>
messenger
=
new
MessagingUdfWithEVsVVWithDegrees
<
K
,
VV
,
Message
,
EV
>(
messagingFunction
,
messageTypeInfo
);
messages
=
this
.
edgesWithValue
.
coGroup
(
iteration
.
getWorkset
()).
where
(
whereArg
)
...
...
@@ -626,34 +626,34 @@ public class ScatterGatherIteration<K, VV, Message, EV>
this
.
updateFunction
.
setOptDegrees
(
this
.
configuration
.
isOptDegrees
());
DataSet
<
Tuple2
<
K
,
Long
>>
inDegrees
=
graph
.
inDegrees
();
DataSet
<
Tuple2
<
K
,
Long
>>
outDegrees
=
graph
.
outDegrees
();
DataSet
<
Tuple2
<
K
,
Long
Value
>>
inDegrees
=
graph
.
inDegrees
();
DataSet
<
Tuple2
<
K
,
Long
Value
>>
outDegrees
=
graph
.
outDegrees
();
DataSet
<
Tuple3
<
K
,
Long
,
Long
>>
degrees
=
inDegrees
.
join
(
outDegrees
).
where
(
0
).
equalTo
(
0
)
.
with
(
new
FlatJoinFunction
<
Tuple2
<
K
,
Long
>,
Tuple2
<
K
,
Long
>,
Tuple3
<
K
,
Long
,
Long
>>()
{
DataSet
<
Tuple3
<
K
,
Long
Value
,
LongValue
>>
degrees
=
inDegrees
.
join
(
outDegrees
).
where
(
0
).
equalTo
(
0
)
.
with
(
new
FlatJoinFunction
<
Tuple2
<
K
,
Long
Value
>,
Tuple2
<
K
,
LongValue
>,
Tuple3
<
K
,
LongValue
,
LongValue
>>()
{
@Override
public
void
join
(
Tuple2
<
K
,
Long
>
first
,
Tuple2
<
K
,
Long
>
second
,
Collector
<
Tuple3
<
K
,
Long
,
Long
>>
out
)
{
out
.
collect
(
new
Tuple3
<
K
,
Long
,
Long
>(
first
.
f0
,
first
.
f1
,
second
.
f1
));
public
void
join
(
Tuple2
<
K
,
Long
Value
>
first
,
Tuple2
<
K
,
LongValue
>
second
,
Collector
<
Tuple3
<
K
,
LongValue
,
LongValue
>>
out
)
{
out
.
collect
(
new
Tuple3
<
K
,
Long
Value
,
LongValue
>(
first
.
f0
,
first
.
f1
,
second
.
f1
));
}
}).
withForwardedFieldsFirst
(
"f0;f1"
).
withForwardedFieldsSecond
(
"f1"
);
DataSet
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>>
verticesWithDegrees
=
initialVertices
DataSet
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>>
verticesWithDegrees
=
initialVertices
.
join
(
degrees
).
where
(
0
).
equalTo
(
0
)
.
with
(
new
FlatJoinFunction
<
Vertex
<
K
,
VV
>,
Tuple3
<
K
,
Long
,
Long
>,
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>>()
{
.
with
(
new
FlatJoinFunction
<
Vertex
<
K
,
VV
>,
Tuple3
<
K
,
LongValue
,
LongValue
>,
Vertex
<
K
,
Tuple3
<
VV
,
LongValue
,
LongValue
>>>()
{
@Override
public
void
join
(
Vertex
<
K
,
VV
>
vertex
,
Tuple3
<
K
,
Long
,
Long
>
degrees
,
Collector
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>>
out
)
throws
Exception
{
public
void
join
(
Vertex
<
K
,
VV
>
vertex
,
Tuple3
<
K
,
Long
Value
,
LongValue
>
degrees
,
Collector
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>>
out
)
throws
Exception
{
out
.
collect
(
new
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>(
vertex
.
getId
(),
new
Tuple3
<
VV
,
Long
,
Long
>(
vertex
.
getValue
(),
degrees
.
f1
,
degrees
.
f2
)));
out
.
collect
(
new
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>(
vertex
.
getId
(),
new
Tuple3
<
VV
,
Long
Value
,
LongValue
>(
vertex
.
getValue
(),
degrees
.
f1
,
degrees
.
f2
)));
}
}).
withForwardedFieldsFirst
(
"f0"
);
// add type info
TypeInformation
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>>
vertexTypes
=
verticesWithDegrees
.
getType
();
TypeInformation
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>>
vertexTypes
=
verticesWithDegrees
.
getType
();
final
DeltaIteration
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>,
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>>
iteration
=
final
DeltaIteration
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>,
Vertex
<
K
,
Tuple3
<
VV
,
LongValue
,
LongValue
>>>
iteration
=
verticesWithDegrees
.
iterateDelta
(
verticesWithDegrees
,
this
.
maximumNumberOfIterations
,
0
);
setUpIteration
(
iteration
);
...
...
@@ -673,11 +673,11 @@ public class ScatterGatherIteration<K, VV, Message, EV>
}
@SuppressWarnings
({
"unchecked"
,
"rawtypes"
})
VertexUpdateUdf
<
K
,
Tuple3
<
VV
,
Long
,
Long
>,
Message
>
updateUdf
=
VertexUpdateUdf
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>,
Message
>
updateUdf
=
new
VertexUpdateUdfVVWithDegrees
(
updateFunction
,
vertexTypes
);
// build the update function (co group)
CoGroupOperator
<?,
?,
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>>
updates
=
CoGroupOperator
<?,
?,
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>>
updates
=
messages
.
coGroup
(
iteration
.
getSolutionSet
()).
where
(
0
).
equalTo
(
0
).
with
(
updateUdf
);
if
(
this
.
configuration
!=
null
&&
this
.
configuration
.
isOptNumVertices
())
{
...
...
@@ -687,9 +687,9 @@ public class ScatterGatherIteration<K, VV, Message, EV>
configureUpdateFunction
(
updates
);
return
iteration
.
closeWith
(
updates
,
updates
).
map
(
new
MapFunction
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>,
Vertex
<
K
,
VV
>>()
{
new
MapFunction
<
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>,
Vertex
<
K
,
VV
>>()
{
public
Vertex
<
K
,
VV
>
map
(
Vertex
<
K
,
Tuple3
<
VV
,
Long
,
Long
>>
vertex
)
{
public
Vertex
<
K
,
VV
>
map
(
Vertex
<
K
,
Tuple3
<
VV
,
Long
Value
,
LongValue
>>
vertex
)
{
return
new
Vertex
<
K
,
VV
>(
vertex
.
getId
(),
vertex
.
getValue
().
f0
);
}
});
...
...
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/generator/CompleteGraphTest.java
浏览文件 @
40749ddc
...
...
@@ -56,10 +56,10 @@ extends AbstractGraphTest {
assertEquals
(
vertexCount
,
graph
.
numberOfVertices
());
assertEquals
(
vertexCount
*(
vertexCount
-
1
),
graph
.
numberOfEdges
());
long
minInDegree
=
graph
.
inDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
;
long
minOutDegree
=
graph
.
outDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
;
long
maxInDegree
=
graph
.
inDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
;
long
maxOutDegree
=
graph
.
outDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
;
long
minInDegree
=
graph
.
inDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
minOutDegree
=
graph
.
outDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
maxInDegree
=
graph
.
inDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
maxOutDegree
=
graph
.
outDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
assertEquals
(
vertexCount
-
1
,
minInDegree
);
assertEquals
(
vertexCount
-
1
,
minOutDegree
);
...
...
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/generator/CycleGraphTest.java
浏览文件 @
40749ddc
...
...
@@ -55,10 +55,10 @@ extends AbstractGraphTest {
assertEquals
(
vertexCount
,
graph
.
numberOfVertices
());
assertEquals
(
2
*
vertexCount
,
graph
.
numberOfEdges
());
long
minInDegree
=
graph
.
inDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
;
long
minOutDegree
=
graph
.
outDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
;
long
maxInDegree
=
graph
.
inDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
;
long
maxOutDegree
=
graph
.
outDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
;
long
minInDegree
=
graph
.
inDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
minOutDegree
=
graph
.
outDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
maxInDegree
=
graph
.
inDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
maxOutDegree
=
graph
.
outDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
assertEquals
(
2
,
minInDegree
);
assertEquals
(
2
,
minOutDegree
);
...
...
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/generator/EmptyGraphTest.java
浏览文件 @
40749ddc
...
...
@@ -54,8 +54,8 @@ extends AbstractGraphTest {
assertEquals
(
vertexCount
,
graph
.
numberOfVertices
());
assertEquals
(
0
,
graph
.
numberOfEdges
());
long
maxInDegree
=
graph
.
inDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
;
long
maxOutDegree
=
graph
.
outDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
;
long
maxInDegree
=
graph
.
inDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
maxOutDegree
=
graph
.
outDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
assertEquals
(
0
,
maxInDegree
);
assertEquals
(
0
,
maxOutDegree
);
...
...
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/generator/GridGraphTest.java
浏览文件 @
40749ddc
...
...
@@ -63,10 +63,10 @@ extends AbstractGraphTest {
assertEquals
(
2
*
3
*
5
*
7
,
graph
.
numberOfVertices
());
assertEquals
(
7
*
2
*
3
*
5
*
7
,
graph
.
numberOfEdges
());
long
minInDegree
=
graph
.
inDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
;
long
minOutDegree
=
graph
.
outDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
;
long
maxInDegree
=
graph
.
inDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
;
long
maxOutDegree
=
graph
.
outDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
;
long
minInDegree
=
graph
.
inDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
minOutDegree
=
graph
.
outDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
maxInDegree
=
graph
.
inDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
maxOutDegree
=
graph
.
outDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
assertEquals
(
7
,
minInDegree
);
assertEquals
(
7
,
minOutDegree
);
...
...
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/generator/HypercubeGraphTest.java
浏览文件 @
40749ddc
...
...
@@ -57,10 +57,10 @@ extends AbstractGraphTest {
assertEquals
(
1L
<<
dimensions
,
graph
.
numberOfVertices
());
assertEquals
(
dimensions
*
(
1L
<<
dimensions
),
graph
.
numberOfEdges
());
long
minInDegree
=
graph
.
inDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
;
long
minOutDegree
=
graph
.
outDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
;
long
maxInDegree
=
graph
.
inDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
;
long
maxOutDegree
=
graph
.
outDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
;
long
minInDegree
=
graph
.
inDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
minOutDegree
=
graph
.
outDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
maxInDegree
=
graph
.
inDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
maxOutDegree
=
graph
.
outDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
assertEquals
(
dimensions
,
minInDegree
);
assertEquals
(
dimensions
,
minOutDegree
);
...
...
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/generator/PathGraphTest.java
浏览文件 @
40749ddc
...
...
@@ -55,10 +55,10 @@ extends AbstractGraphTest {
assertEquals
(
vertexCount
,
graph
.
numberOfVertices
());
assertEquals
(
2
*
(
vertexCount
-
1
),
graph
.
numberOfEdges
());
long
minInDegree
=
graph
.
inDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
;
long
minOutDegree
=
graph
.
outDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
;
long
maxInDegree
=
graph
.
inDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
;
long
maxOutDegree
=
graph
.
outDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
;
long
minInDegree
=
graph
.
inDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
minOutDegree
=
graph
.
outDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
maxInDegree
=
graph
.
inDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
maxOutDegree
=
graph
.
outDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
assertEquals
(
1
,
minInDegree
);
assertEquals
(
1
,
minOutDegree
);
...
...
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/generator/SingletonEdgeGraphTest.java
浏览文件 @
40749ddc
...
...
@@ -56,10 +56,10 @@ extends AbstractGraphTest {
assertEquals
(
2
*
vertexPairCount
,
graph
.
numberOfVertices
());
assertEquals
(
2
*
vertexPairCount
,
graph
.
numberOfEdges
());
long
minInDegree
=
graph
.
inDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
;
long
minOutDegree
=
graph
.
outDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
;
long
maxInDegree
=
graph
.
inDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
;
long
maxOutDegree
=
graph
.
outDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
;
long
minInDegree
=
graph
.
inDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
minOutDegree
=
graph
.
outDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
maxInDegree
=
graph
.
inDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
maxOutDegree
=
graph
.
outDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
assertEquals
(
1
,
minInDegree
);
assertEquals
(
1
,
minOutDegree
);
...
...
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/generator/StarGraphTest.java
浏览文件 @
40749ddc
...
...
@@ -57,10 +57,10 @@ extends AbstractGraphTest {
assertEquals
(
vertexCount
,
graph
.
numberOfVertices
());
assertEquals
(
2
*
(
vertexCount
-
1
),
graph
.
numberOfEdges
());
long
minInDegree
=
graph
.
inDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
;
long
minOutDegree
=
graph
.
outDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
;
long
maxInDegree
=
graph
.
inDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
;
long
maxOutDegree
=
graph
.
outDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
;
long
minInDegree
=
graph
.
inDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
minOutDegree
=
graph
.
outDegrees
().
min
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
maxInDegree
=
graph
.
inDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
long
maxOutDegree
=
graph
.
outDegrees
().
max
(
1
).
collect
().
get
(
0
).
f1
.
getValue
()
;
assertEquals
(
1
,
minInDegree
);
assertEquals
(
1
,
minOutDegree
);
...
...
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/DegreesITCase.java
浏览文件 @
40749ddc
...
...
@@ -25,6 +25,7 @@ import org.apache.flink.api.java.tuple.Tuple2;
import
org.apache.flink.graph.Graph
;
import
org.apache.flink.graph.test.TestGraphUtils
;
import
org.apache.flink.test.util.MultipleProgramsTestBase
;
import
org.apache.flink.types.LongValue
;
import
org.apache.flink.types.NullValue
;
import
org.junit.Test
;
import
org.junit.runner.RunWith
;
...
...
@@ -50,8 +51,8 @@ public class DegreesITCase extends MultipleProgramsTestBase {
Graph
<
Long
,
Long
,
Long
>
graph
=
Graph
.
fromDataSet
(
TestGraphUtils
.
getLongLongVertexData
(
env
),
TestGraphUtils
.
getLongLongEdgeData
(
env
),
env
);
DataSet
<
Tuple2
<
Long
,
Long
>>
data
=
graph
.
outDegrees
();
List
<
Tuple2
<
Long
,
Long
>>
result
=
data
.
collect
();
DataSet
<
Tuple2
<
Long
,
LongValue
>>
data
=
graph
.
outDegrees
();
List
<
Tuple2
<
Long
,
LongValue
>>
result
=
data
.
collect
();
expectedResult
=
"1,2\n"
+
...
...
@@ -76,8 +77,8 @@ public class DegreesITCase extends MultipleProgramsTestBase {
DataSet
<
Tuple2
<
Long
,
Long
>>
data
=
graph
.
outDegrees
();
List
<
Tuple2
<
Long
,
Long
>>
result
=
data
.
collect
();
DataSet
<
Tuple2
<
Long
,
LongValue
>>
data
=
graph
.
outDegrees
();
List
<
Tuple2
<
Long
,
LongValue
>>
result
=
data
.
collect
();
expectedResult
=
"1,3\n"
+
"2,1\n"
+
...
...
@@ -99,8 +100,8 @@ public class DegreesITCase extends MultipleProgramsTestBase {
TestGraphUtils
.
getLongLongEdgeData
(
env
),
env
);
DataSet
<
Tuple2
<
Long
,
Long
>>
data
=
graph
.
inDegrees
();
List
<
Tuple2
<
Long
,
Long
>>
result
=
data
.
collect
();
DataSet
<
Tuple2
<
Long
,
LongValue
>>
data
=
graph
.
inDegrees
();
List
<
Tuple2
<
Long
,
LongValue
>>
result
=
data
.
collect
();
expectedResult
=
"1,1\n"
+
"2,1\n"
+
...
...
@@ -120,8 +121,8 @@ public class DegreesITCase extends MultipleProgramsTestBase {
Graph
<
Long
,
Long
,
Long
>
graph
=
Graph
.
fromDataSet
(
TestGraphUtils
.
getLongLongVertexData
(
env
),
TestGraphUtils
.
getLongLongEdgeDataWithZeroDegree
(
env
),
env
);
DataSet
<
Tuple2
<
Long
,
Long
>>
data
=
graph
.
inDegrees
();
List
<
Tuple2
<
Long
,
Long
>>
result
=
data
.
collect
();
DataSet
<
Tuple2
<
Long
,
LongValue
>>
data
=
graph
.
inDegrees
();
List
<
Tuple2
<
Long
,
LongValue
>>
result
=
data
.
collect
();
expectedResult
=
"1,0\n"
+
"2,1\n"
+
...
...
@@ -142,8 +143,8 @@ public class DegreesITCase extends MultipleProgramsTestBase {
Graph
<
Long
,
Long
,
Long
>
graph
=
Graph
.
fromDataSet
(
TestGraphUtils
.
getLongLongVertexData
(
env
),
TestGraphUtils
.
getLongLongEdgeData
(
env
),
env
);
DataSet
<
Tuple2
<
Long
,
Long
>>
data
=
graph
.
getDegrees
();
List
<
Tuple2
<
Long
,
Long
>>
result
=
data
.
collect
();
DataSet
<
Tuple2
<
Long
,
LongValue
>>
data
=
graph
.
getDegrees
();
List
<
Tuple2
<
Long
,
LongValue
>>
result
=
data
.
collect
();
expectedResult
=
"1,3\n"
+
"2,2\n"
+
...
...
@@ -164,8 +165,8 @@ public class DegreesITCase extends MultipleProgramsTestBase {
Graph
<
Long
,
NullValue
,
Long
>
graph
=
Graph
.
fromDataSet
(
TestGraphUtils
.
getDisconnectedLongLongEdgeData
(
env
),
env
);
DataSet
<
Tuple2
<
Long
,
Long
>>
data
=
graph
.
outDegrees
();
List
<
Tuple2
<
Long
,
Long
>>
result
=
data
.
collect
();
DataSet
<
Tuple2
<
Long
,
LongValue
>>
data
=
graph
.
outDegrees
();
List
<
Tuple2
<
Long
,
LongValue
>>
result
=
data
.
collect
();
expectedResult
=
"1,2\n"
+
"2,1\n"
+
...
...
flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/DegreesWithExceptionITCase.java
浏览文件 @
40749ddc
...
...
@@ -28,6 +28,7 @@ import org.apache.flink.graph.Graph;
import
org.apache.flink.graph.test.TestGraphUtils
;
import
org.apache.flink.test.util.ForkableFlinkMiniCluster
;
import
org.apache.flink.types.LongValue
;
import
org.junit.AfterClass
;
import
org.junit.BeforeClass
;
import
org.junit.Test
;
...
...
@@ -81,7 +82,7 @@ public class DegreesWithExceptionITCase {
TestGraphUtils
.
getLongLongEdgeInvalidSrcData
(
env
),
env
);
try
{
graph
.
outDegrees
().
output
(
new
DiscardingOutputFormat
<
Tuple2
<
Long
,
Long
>>());
graph
.
outDegrees
().
output
(
new
DiscardingOutputFormat
<
Tuple2
<
Long
,
Long
Value
>>());
env
.
execute
();
fail
(
"graph.outDegrees() did not fail."
);
...
...
@@ -105,7 +106,7 @@ public class DegreesWithExceptionITCase {
TestGraphUtils
.
getLongLongEdgeInvalidTrgData
(
env
),
env
);
try
{
graph
.
inDegrees
().
output
(
new
DiscardingOutputFormat
<
Tuple2
<
Long
,
Long
>>());
graph
.
inDegrees
().
output
(
new
DiscardingOutputFormat
<
Tuple2
<
Long
,
Long
Value
>>());
env
.
execute
();
fail
(
"graph.inDegrees() did not fail."
);
...
...
@@ -129,7 +130,7 @@ public class DegreesWithExceptionITCase {
TestGraphUtils
.
getLongLongEdgeInvalidTrgData
(
env
),
env
);
try
{
graph
.
getDegrees
().
output
(
new
DiscardingOutputFormat
<
Tuple2
<
Long
,
Long
>>());
graph
.
getDegrees
().
output
(
new
DiscardingOutputFormat
<
Tuple2
<
Long
,
Long
Value
>>());
env
.
execute
();
fail
(
"graph.getDegrees() did not fail."
);
...
...
@@ -153,7 +154,7 @@ public class DegreesWithExceptionITCase {
TestGraphUtils
.
getLongLongEdgeInvalidSrcData
(
env
),
env
);
try
{
graph
.
getDegrees
().
output
(
new
DiscardingOutputFormat
<
Tuple2
<
Long
,
Long
>>());
graph
.
getDegrees
().
output
(
new
DiscardingOutputFormat
<
Tuple2
<
Long
,
Long
Value
>>());
env
.
execute
();
fail
(
"graph.getDegrees() did not fail."
);
...
...
@@ -177,7 +178,7 @@ public class DegreesWithExceptionITCase {
TestGraphUtils
.
getLongLongEdgeInvalidSrcTrgData
(
env
),
env
);
try
{
graph
.
getDegrees
().
output
(
new
DiscardingOutputFormat
<
Tuple2
<
Long
,
Long
>>());
graph
.
getDegrees
().
output
(
new
DiscardingOutputFormat
<
Tuple2
<
Long
,
Long
Value
>>());
env
.
execute
();
fail
(
"graph.getDegrees() did not fail."
);
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录