Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
468bcb0f
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
468bcb0f
编写于
5月 19, 2014
作者:
F
Fabian Hueske
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Improved comments in Java example jobs
上级
2bb3e982
变更
15
隐藏空白更改
内联
并排
Showing
15 changed file
with
80 addition
and
19 deletion
+80
-19
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/clustering/KMeans.java
.../java/eu/stratosphere/example/java/clustering/KMeans.java
+7
-3
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/clustering/util/KMeansData.java
...stratosphere/example/java/clustering/util/KMeansData.java
+5
-0
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/graph/ConnectedComponents.java
.../stratosphere/example/java/graph/ConnectedComponents.java
+5
-1
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/graph/EnumTrianglesBasic.java
...u/stratosphere/example/java/graph/EnumTrianglesBasic.java
+3
-0
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/graph/EnumTrianglesOpt.java
.../eu/stratosphere/example/java/graph/EnumTrianglesOpt.java
+3
-0
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/graph/PageRankBasic.java
...ava/eu/stratosphere/example/java/graph/PageRankBasic.java
+6
-2
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/graph/util/ConnectedComponentsData.java
...here/example/java/graph/util/ConnectedComponentsData.java
+5
-0
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/graph/util/EnumTrianglesData.java
...ratosphere/example/java/graph/util/EnumTrianglesData.java
+5
-0
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/graph/util/PageRankData.java
...eu/stratosphere/example/java/graph/util/PageRankData.java
+5
-0
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/relational/TPCHQuery10.java
.../eu/stratosphere/example/java/relational/TPCHQuery10.java
+3
-0
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/relational/TPCHQuery3.java
...a/eu/stratosphere/example/java/relational/TPCHQuery3.java
+4
-1
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/relational/WebLogAnalysis.java
.../stratosphere/example/java/relational/WebLogAnalysis.java
+15
-12
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/relational/util/WebLogData.java
...stratosphere/example/java/relational/util/WebLogData.java
+5
-0
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/wordcount/WordCount.java
...ava/eu/stratosphere/example/java/wordcount/WordCount.java
+4
-0
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/wordcount/util/WordCountData.java
...ratosphere/example/java/wordcount/util/WordCountData.java
+5
-0
未找到文件。
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/clustering/KMeans.java
浏览文件 @
468bcb0f
...
...
@@ -57,6 +57,10 @@ import eu.stratosphere.example.java.clustering.util.KMeansData;
* </ul>
*
* <p>
* Usage: <code>KMeans <points path> <centers path> <result path> <num iterations></code><br>
* If no parameters are provided, the program is run with default data from {@link KMeansData} and 10 iterations.
*
* <p>
* This example shows how to use:
* <ul>
* <li>Bulk iterations
...
...
@@ -103,7 +107,7 @@ public class KMeans {
// emit result
if
(
fileOutput
)
{
clusteredPoints
.
writeAsCsv
(
outputPath
,
"\n"
,
"
,
"
);
clusteredPoints
.
writeAsCsv
(
outputPath
,
"\n"
,
"
"
);
}
else
{
clusteredPoints
.
print
();
}
...
...
@@ -153,7 +157,7 @@ public class KMeans {
@Override
public
String
toString
()
{
return
x
+
"
,
"
+
y
;
return
x
+
"
"
+
y
;
}
}
...
...
@@ -178,7 +182,7 @@ public class KMeans {
@Override
public
String
toString
()
{
return
id
+
"
,
"
+
super
.
toString
();
return
id
+
"
"
+
super
.
toString
();
}
}
...
...
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/clustering/util/KMeansData.java
浏览文件 @
468bcb0f
...
...
@@ -18,6 +18,11 @@ import eu.stratosphere.api.java.ExecutionEnvironment;
import
eu.stratosphere.example.java.clustering.KMeans.Centroid
;
import
eu.stratosphere.example.java.clustering.KMeans.Point
;
/**
* Provides the default data sets used for the K-Means example program.
* The default data sets are used, if no parameters are given to the program.
*
*/
public
class
KMeansData
{
public
static
DataSet
<
Centroid
>
getDefaultCentroidDataSet
(
ExecutionEnvironment
env
)
{
...
...
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/graph/ConnectedComponents.java
浏览文件 @
468bcb0f
...
...
@@ -55,6 +55,10 @@ import eu.stratosphere.util.Collector;
* </ul>
*
* <p>
* Usage: <code>ConnectedComponents <vertices path> <edges path> <result path> <max number of iterations></code><br>
* If no parameters are provided, the program is run with default data from {@link ConnectedComponentsData} and 10 iterations.
*
* <p>
* This example shows how to use:
* <ul>
* <li>Delta Iterations
...
...
@@ -79,7 +83,7 @@ public class ConnectedComponents implements ProgramDescription {
DataSet
<
Long
>
vertices
=
getVertexDataSet
(
env
);
DataSet
<
Tuple2
<
Long
,
Long
>>
edges
=
getEdgeDataSet
(
env
);
// assign the initial components (equal to the vertex id
.
// assign the initial components (equal to the vertex id
)
DataSet
<
Tuple2
<
Long
,
Long
>>
verticesWithInitialId
=
vertices
.
map
(
new
DuplicateValue
<
Long
>());
// open a delta iteration
...
...
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/graph/EnumTrianglesBasic.java
浏览文件 @
468bcb0f
...
...
@@ -54,6 +54,9 @@ import eu.stratosphere.util.Collector;
* (2)-(12)
* </pre>
*
* Usage: <code>EnumTriangleBasic <edge path> <result path></code><br>
* If no parameters are provided, the program is run with default data from {@link EnumTrianglesData}.
*
* <p>
* This example shows how to use:
* <ul>
...
...
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/graph/EnumTrianglesOpt.java
浏览文件 @
468bcb0f
...
...
@@ -64,6 +64,9 @@ import eu.stratosphere.util.Collector;
* (2)-(12)
* </pre>
*
* Usage: <code>EnumTriangleOpt <edge path> <result path></code><br>
* If no parameters are provided, the program is run with default data from {@link EnumTrianglesData}.
*
* <p>
* This example shows how to use:
* <ul>
...
...
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/graph/PageRankBasic.java
浏览文件 @
468bcb0f
...
...
@@ -55,6 +55,10 @@ import eu.stratosphere.util.Collector;
* </ul>
*
* <p>
* Usage: <code>PageRankBasic <vertices with initial ranks path> <edges path> <output path> <num vertices> <num iterations></code><br>
* If no parameters are provided, the program is run with default data from {@link PageRankData} and 10 iterations.
*
* <p>
* This example shows how to use:
* <ul>
* <li>Bulk Iterations
...
...
@@ -211,14 +215,14 @@ public class PageRankBasic {
numVertices
=
Integer
.
parseInt
(
args
[
3
]);
maxIterations
=
Integer
.
parseInt
(
args
[
4
]);
}
else
{
System
.
err
.
println
(
"Usage: PageRankBasic <vert
ex with initial rank input
> <edges path> <output path> <num vertices> <num iterations>"
);
System
.
err
.
println
(
"Usage: PageRankBasic <vert
ices with initial ranks path
> <edges path> <output path> <num vertices> <num iterations>"
);
System
.
exit
(
1
);
}
}
else
{
System
.
out
.
println
(
"Executing PageRank Basic example with default parameters and built-in default data."
);
System
.
out
.
println
(
" Provide parameters to read input data from files."
);
System
.
out
.
println
(
" See the documentation for the correct format of input files."
);
System
.
out
.
println
(
" Usage: PageRankBasic <vert
ex with initial rank input
> <edges path> <output path> <num vertices> <num iterations>"
);
System
.
out
.
println
(
" Usage: PageRankBasic <vert
ices with initial ranks path
> <edges path> <output path> <num vertices> <num iterations>"
);
numVertices
=
PageRankData
.
getNumberOfPages
();
}
...
...
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/graph/util/ConnectedComponentsData.java
浏览文件 @
468bcb0f
...
...
@@ -20,6 +20,11 @@ import eu.stratosphere.api.java.DataSet;
import
eu.stratosphere.api.java.ExecutionEnvironment
;
import
eu.stratosphere.api.java.tuple.Tuple2
;
/**
* Provides the default data sets used for the Connected Components example program.
* The default data sets are used, if no parameters are given to the program.
*
*/
public
class
ConnectedComponentsData
{
public
static
DataSet
<
Long
>
getDefaultVertexDataSet
(
ExecutionEnvironment
env
)
{
...
...
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/graph/util/EnumTrianglesData.java
浏览文件 @
468bcb0f
...
...
@@ -19,6 +19,11 @@ import eu.stratosphere.api.java.DataSet;
import
eu.stratosphere.api.java.ExecutionEnvironment
;
import
eu.stratosphere.example.java.graph.util.EnumTrianglesDataTypes.Edge
;
/**
* Provides the default data sets used for the Triangle Enumeration example programs.
* The default data sets are used, if no parameters are given to the program.
*
*/
public
class
EnumTrianglesData
{
public
static
DataSet
<
Edge
>
getDefaultEdgeDataSet
(
ExecutionEnvironment
env
)
{
...
...
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/graph/util/PageRankData.java
浏览文件 @
468bcb0f
...
...
@@ -22,6 +22,11 @@ import eu.stratosphere.api.java.DataSet;
import
eu.stratosphere.api.java.ExecutionEnvironment
;
import
eu.stratosphere.api.java.tuple.Tuple2
;
/**
* Provides the default data sets used for the PageRank example program.
* The default data sets are used, if no parameters are given to the program.
*
*/
public
class
PageRankData
{
private
static
int
numPages
=
15
;
...
...
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/relational/TPCHQuery10.java
浏览文件 @
468bcb0f
...
...
@@ -71,6 +71,9 @@ import eu.stratosphere.api.java.tuple.Tuple6;
* as generated by the TPC-H data generator which is available at <a href="http://www.tpc.org/tpch/">http://www.tpc.org/tpch/</a>.
*
* <p>
* Usage: <code>TPCHQuery10 <customer-csv path> <orders-csv path> <lineitem-csv path> <nation-csv path> <result path></code><br>
*
* <p>
* This example shows how to use:
* <ul>
* <li> tuple data types
...
...
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/relational/TPCHQuery3.java
浏览文件 @
468bcb0f
...
...
@@ -69,6 +69,9 @@ import eu.stratosphere.api.java.tuple.Tuple5;
* <p>
* Input files are plain text CSV files using the pipe character ('|') as field separator
* as generated by the TPC-H data generator which is available at <a href="http://www.tpc.org/tpch/">http://www.tpc.org/tpch/</a>.
*
* <p>
* Usage: <code>TPCHQuery3 <lineitem-csv path> <customer-csv path> <orders-csv path> <result path></code><br>
*
* <p>
* This example shows how to use:
...
...
@@ -157,7 +160,7 @@ public class TPCHQuery3 {
}
});
// Join the last join result with Line
I
tems
// Join the last join result with Line
i
tems
DataSet
<
ShippingPriorityItem
>
joined
=
customerWithOrders
.
join
(
li
)
.
where
(
4
)
...
...
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/relational/WebLogAnalysis.java
浏览文件 @
468bcb0f
...
...
@@ -74,6 +74,10 @@ import eu.stratosphere.util.Collector;
* </pre></code>
*
* <p>
* Usage: <code>WebLogAnalysis <documents path> <ranks path> <visits path> <result path></code><br>
* If no parameters are provided, the program is run with default data from {@link WebLogData}.
*
* <p>
* This example shows how to use:
* <ul>
* <li> tuple data types
...
...
@@ -100,29 +104,28 @@ public class WebLogAnalysis {
DataSet
<
Tuple3
<
Integer
,
String
,
Integer
>>
ranks
=
getRanksDataSet
(
env
);
DataSet
<
Tuple2
<
String
,
String
>>
visits
=
getVisitsDataSet
(
env
);
//
Create DataSet for filtering the entries from the documents relation
//
Retain documents with keywords
DataSet
<
Tuple1
<
String
>>
filterDocs
=
documents
.
filter
(
new
FilterDocs
())
.
filter
(
new
FilterDoc
ByKeyWord
s
())
.
project
(
0
).
types
(
String
.
class
);
//
Create DataSet for filtering the entries from the ranks relation
//
Filter ranks by minimum rank
DataSet
<
Tuple3
<
Integer
,
String
,
Integer
>>
filterRanks
=
ranks
.
filter
(
new
Filter
Ranks
());
.
filter
(
new
Filter
ByRank
());
//
Create DataSet for filtering the entries from the visits relation
//
Filter visits by visit date
DataSet
<
Tuple1
<
String
>>
filterVisits
=
visits
.
filter
(
new
FilterVisits
())
.
filter
(
new
FilterVisits
ByDate
())
.
project
(
0
).
types
(
String
.
class
);
//
Create DataSet to join the filtered documents and ranks relation
//
Join the filtered documents and ranks, i.e., get all URLs with min rank and keywords
DataSet
<
Tuple3
<
Integer
,
String
,
Integer
>>
joinDocsRanks
=
filterDocs
.
join
(
filterRanks
)
.
where
(
0
).
equalTo
(
1
)
.
projectSecond
(
0
,
1
,
2
)
.
types
(
Integer
.
class
,
String
.
class
,
Integer
.
class
);
// Create DataSet to realize a anti join between the joined
// documents and ranks relation and the filtered visits relation
// Anti-join urls with visits, i.e., retain all URLs which have NOT been visited in a certain time
DataSet
<
Tuple3
<
Integer
,
String
,
Integer
>>
result
=
joinDocsRanks
.
coGroup
(
filterVisits
)
.
where
(
1
).
equalTo
(
0
)
...
...
@@ -148,7 +151,7 @@ public class WebLogAnalysis {
* MapFunction that filters for documents that contain a certain set of
* keywords.
*/
public
static
class
FilterDocs
extends
FilterFunction
<
Tuple2
<
String
,
String
>>
{
public
static
class
FilterDoc
ByKeyWord
s
extends
FilterFunction
<
Tuple2
<
String
,
String
>>
{
private
static
final
String
[]
KEYWORDS
=
{
" editors "
,
" oscillations "
};
...
...
@@ -176,7 +179,7 @@ public class WebLogAnalysis {
/**
* MapFunction that filters for records where the rank exceeds a certain threshold.
*/
public
static
class
Filter
Ranks
extends
FilterFunction
<
Tuple3
<
Integer
,
String
,
Integer
>>
{
public
static
class
Filter
ByRank
extends
FilterFunction
<
Tuple3
<
Integer
,
String
,
Integer
>>
{
private
static
final
int
RANKFILTER
=
40
;
...
...
@@ -199,7 +202,7 @@ public class WebLogAnalysis {
* MapFunction that filters for records of the visits relation where the year
* (from the date string) is equal to a certain value.
*/
public
static
class
FilterVisits
extends
FilterFunction
<
Tuple2
<
String
,
String
>>
{
public
static
class
FilterVisits
ByDate
extends
FilterFunction
<
Tuple2
<
String
,
String
>>
{
private
static
final
int
YEARFILTER
=
2007
;
...
...
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/relational/util/WebLogData.java
浏览文件 @
468bcb0f
...
...
@@ -23,6 +23,11 @@ import eu.stratosphere.api.java.ExecutionEnvironment;
import
eu.stratosphere.api.java.tuple.Tuple2
;
import
eu.stratosphere.api.java.tuple.Tuple3
;
/**
* Provides the default data sets used for the Weblog Analysis example program.
* The default data sets are used, if no parameters are given to the program.
*
*/
public
class
WebLogData
{
public
static
DataSet
<
Tuple2
<
String
,
String
>>
getDocumentDataSet
(
ExecutionEnvironment
env
)
{
...
...
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/wordcount/WordCount.java
浏览文件 @
468bcb0f
...
...
@@ -30,6 +30,10 @@ import eu.stratosphere.util.Collector;
* The input is a plain text file with lines separated by newline characters.
*
* <p>
* Usage: <code>WordCount <text path> <result path></code><br>
* If no parameters are provided, the program is run with default data from {@link WordCountData}.
*
* <p>
* This example shows how to:
* <ul>
* <li>write a simple Stratosphere program.
...
...
stratosphere-examples/stratosphere-java-examples/src/main/java/eu/stratosphere/example/java/wordcount/util/WordCountData.java
浏览文件 @
468bcb0f
...
...
@@ -17,6 +17,11 @@ package eu.stratosphere.example.java.wordcount.util;
import
eu.stratosphere.api.java.DataSet
;
import
eu.stratosphere.api.java.ExecutionEnvironment
;
/**
* Provides the default data sets used for the WordCount example program.
* The default data sets are used, if no parameters are given to the program.
*
*/
public
class
WordCountData
{
public
static
DataSet
<
String
>
getDefaultTextLineDataSet
(
ExecutionEnvironment
env
)
{
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录