Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
梦中观雨
cat
提交
ac3a7cd4
C
cat
项目概览
梦中观雨
/
cat
与 Fork 源项目一致
从无法访问的项目Fork
通知
1
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
C
cat
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
ac3a7cd4
编写于
7月 15, 2014
作者:
曾
曾伟伟
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
app data collect
上级
824801a1
变更
15
隐藏空白更改
内联
并排
Showing
15 changed file
with
706 addition
and
194 deletion
+706
-194
cat-broker/pom.xml
cat-broker/pom.xml
+192
-181
cat-broker/src/main/java/com/dianping/cat/broker/api/app/AppDataConsumer.java
...java/com/dianping/cat/broker/api/app/AppDataConsumer.java
+169
-0
cat-broker/src/main/java/com/dianping/cat/broker/api/app/AppDataQueue.java
...in/java/com/dianping/cat/broker/api/app/AppDataQueue.java
+26
-0
cat-broker/src/main/java/com/dianping/cat/broker/api/app/BucketHandler.java
...n/java/com/dianping/cat/broker/api/app/BucketHandler.java
+147
-0
cat-broker/src/main/java/com/dianping/cat/broker/api/page/IpService.java
...main/java/com/dianping/cat/broker/api/page/IpService.java
+33
-9
cat-broker/src/main/java/com/dianping/cat/broker/api/page/batch/Handler.java
.../java/com/dianping/cat/broker/api/page/batch/Handler.java
+63
-3
cat-broker/src/main/java/com/dianping/cat/broker/build/ComponentsConfigurator.java
...com/dianping/cat/broker/build/ComponentsConfigurator.java
+1
-1
cat-broker/src/main/resources/META-INF/dal/model/appData-codegen.xml
...src/main/resources/META-INF/dal/model/appData-codegen.xml
+19
-0
cat-broker/src/main/resources/META-INF/dal/model/appData-manifest.xml
...rc/main/resources/META-INF/dal/model/appData-manifest.xml
+6
-0
cat-broker/src/main/resources/META-INF/dal/model/appData-model.xml
...r/src/main/resources/META-INF/dal/model/appData-model.xml
+3
-0
cat-broker/src/main/resources/META-INF/plexus/components.xml
cat-broker/src/main/resources/META-INF/plexus/components.xml
+25
-0
cat-broker/src/main/resources/META-INF/wizard/model/manifest.xml
...ker/src/main/resources/META-INF/wizard/model/manifest.xml
+5
-0
cat-broker/src/main/resources/META-INF/wizard/model/wizard.xml
...roker/src/main/resources/META-INF/wizard/model/wizard.xml
+6
-0
cat-broker/src/main/webapp/jsp/api/app.jsp
cat-broker/src/main/webapp/jsp/api/app.jsp
+5
-0
cat-home/src/main/webapp/jsp/report/cdn.jsp
cat-home/src/main/webapp/jsp/report/cdn.jsp
+6
-0
未找到文件。
cat-broker/pom.xml
浏览文件 @
ac3a7cd4
<?xml version="1.0" encoding="UTF-8"?>
<?xml version="1.0" encoding="UTF-8"?>
<project
xmlns=
"http://maven.apache.org/POM/4.0.0"
xmlns:xsi=
"http://www.w3.org/2001/XMLSchema-instance"
<project
xmlns=
"http://maven.apache.org/POM/4.0.0"
xmlns:xsi=
"http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation=
"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"
>
xsi:schemaLocation=
"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"
>
<parent>
<parent>
<groupId>
com.dianping.cat
</groupId>
<groupId>
com.dianping.cat
</groupId>
<artifactId>
parent
</artifactId>
<artifactId>
parent
</artifactId>
<version>
1.0.5
</version>
<version>
1.0.5
</version>
</parent>
</parent>
<modelVersion>
4.0.0
</modelVersion>
<modelVersion>
4.0.0
</modelVersion>
<artifactId>
broker-service
</artifactId>
<artifactId>
broker-service
</artifactId>
<name>
broker-service
</name>
<name>
broker-service
</name>
<packaging>
war
</packaging>
<packaging>
war
</packaging>
<dependencies>
<dependencies>
<dependency>
<dependency>
<groupId>
org.unidal.eunit
</groupId>
<groupId>
org.unidal.eunit
</groupId>
<artifactId>
EunitTestFwk
</artifactId>
<artifactId>
EunitTestFwk
</artifactId>
<version>
1.2.1
</version>
<version>
1.2.1
</version>
<scope>
test
</scope>
<scope>
test
</scope>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
org.unidal.framework
</groupId>
<groupId>
org.unidal.framework
</groupId>
<artifactId>
foundation-service
</artifactId>
<artifactId>
foundation-service
</artifactId>
<version>
2.0.5
</version>
<version>
2.0.5
</version>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
com.dianping.cat
</groupId>
<groupId>
com.dianping.cat
</groupId>
<artifactId>
cat-core
</artifactId>
<artifactId>
cat-core
</artifactId>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
com.google.code.gson
</groupId>
<groupId>
com.google.code.gson
</groupId>
<artifactId>
gson
</artifactId>
<artifactId>
gson
</artifactId>
<version>
1.6
</version>
<version>
1.6
</version>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
org.unidal.webres
</groupId>
<groupId>
org.unidal.webres
</groupId>
<artifactId>
WebResServer
</artifactId>
<artifactId>
WebResServer
</artifactId>
<version>
1.2.1
</version>
<version>
1.2.1
</version>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
org.unidal.framework
</groupId>
<groupId>
org.unidal.framework
</groupId>
<artifactId>
web-framework
</artifactId>
<artifactId>
web-framework
</artifactId>
<version>
2.0.5
</version>
<version>
2.0.5
</version>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
javax.servlet
</groupId>
<groupId>
javax.servlet
</groupId>
<artifactId>
servlet-api
</artifactId>
<artifactId>
servlet-api
</artifactId>
<scope>
provided
</scope>
<scope>
provided
</scope>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
javax.servlet
</groupId>
<groupId>
javax.servlet
</groupId>
<artifactId>
jstl
</artifactId>
<artifactId>
jstl
</artifactId>
<version>
1.2
</version>
<version>
1.2
</version>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
org.mortbay.jetty
</groupId>
<groupId>
org.mortbay.jetty
</groupId>
<artifactId>
jsp-api-2.1
</artifactId>
<artifactId>
jsp-api-2.1
</artifactId>
<version>
6.1.14
</version>
<version>
6.1.14
</version>
<scope>
provided
</scope>
<scope>
provided
</scope>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
org.mortbay.jetty
</groupId>
<groupId>
org.mortbay.jetty
</groupId>
<artifactId>
jsp-2.1
</artifactId>
<artifactId>
jsp-2.1
</artifactId>
<scope>
provided
</scope>
<scope>
provided
</scope>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
mysql
</groupId>
<groupId>
mysql
</groupId>
<artifactId>
mysql-connector-java
</artifactId>
<artifactId>
mysql-connector-java
</artifactId>
<scope>
runtime
</scope>
<scope>
runtime
</scope>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
org.freemarker
</groupId>
<groupId>
org.freemarker
</groupId>
<artifactId>
freemarker
</artifactId>
<artifactId>
freemarker
</artifactId>
<version>
2.3.9
</version>
<version>
2.3.9
</version>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
junit
</groupId>
<groupId>
junit
</groupId>
<artifactId>
junit
</artifactId>
<artifactId>
junit
</artifactId>
<scope>
test
</scope>
<scope>
test
</scope>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
org.mortbay.jetty
</groupId>
<groupId>
org.mortbay.jetty
</groupId>
<artifactId>
jetty
</artifactId>
<artifactId>
jetty
</artifactId>
<scope>
test
</scope>
<scope>
test
</scope>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
org.unidal.framework
</groupId>
<groupId>
org.unidal.framework
</groupId>
<artifactId>
test-framework
</artifactId>
<artifactId>
test-framework
</artifactId>
<version>
2.0.4
</version>
<version>
2.0.4
</version>
<scope>
test
</scope>
<scope>
test
</scope>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
org.unidal.framework
</groupId>
<groupId>
org.unidal.framework
</groupId>
<artifactId>
dal-jdbc
</artifactId>
<artifactId>
dal-jdbc
</artifactId>
<version>
2.0.4
</version>
<version>
2.0.4
</version>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
org.apache.commons
</groupId>
<groupId>
org.apache.commons
</groupId>
<artifactId>
commons-email
</artifactId>
<artifactId>
commons-email
</artifactId>
<version>
1.1
</version>
<version>
1.1
</version>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
javax.mail
</groupId>
<groupId>
javax.mail
</groupId>
<artifactId>
mail
</artifactId>
<artifactId>
mail
</artifactId>
<version>
1.4.4
</version>
<version>
1.4.4
</version>
</dependency>
</dependency>
<dependency>
<dependency>
<groupId>
com.google.code.javaparser
</groupId>
<groupId>
com.google.code.javaparser
</groupId>
<artifactId>
javaparser
</artifactId>
<artifactId>
javaparser
</artifactId>
<version>
1.0.8
</version>
<version>
1.0.8
</version>
</dependency>
</dependency>
</dependencies>
</dependencies>
<build>
<build>
<resources>
<resources>
<resource>
<resource>
<directory>
${basedir}/src/main/resources
</directory>
<directory>
${basedir}/src/main/resources
</directory>
<filtering>
true
</filtering>
<filtering>
true
</filtering>
</resource>
</resource>
</resources>
</resources>
<plugins>
<plugins>
<plugin>
<plugin>
<groupId>
org.unidal.maven.plugins
</groupId>
<groupId>
org.unidal.maven.plugins
</groupId>
<artifactId>
codegen-maven-plugin
</artifactId>
<artifactId>
codegen-maven-plugin
</artifactId>
<version>
2.0.9
</version>
<version>
2.0.9
</version>
<executions>
<executions>
<execution>
<execution>
<id>
generate plexus component descriptor
</id>
<id>
generate plexus component descriptor
</id>
<phase>
process-classes
</phase>
<phase>
process-classes
</phase>
<goals>
<goals>
<goal>
plexus
</goal>
<goal>
plexus
</goal>
</goals>
</goals>
<configuration>
<configuration>
<className>
com.dianping.cat.broker.build.ComponentsConfigurator
</className>
<className>
com.dianping.cat.broker.build.ComponentsConfigurator
</className>
<env>
dev
</env>
<env>
dev
</env>
</configuration>
</configuration>
</execution>
</execution>
<execution>
</executions>
<id>
generate dal model files
</id>
</plugin>
<phase>
generate-sources
</phase>
<plugin>
<goals>
<groupId>
org.apache.maven.plugins
</groupId>
<goal>
dal-model
</goal>
<artifactId>
maven-war-plugin
</artifactId>
</goals>
<version>
2.1.1
</version>
<configuration>
<configuration>
<manifest>
<![CDATA[
<warName>
broker-service-${env}-${version}
</warName>
${basedir}/src/main/resources/META-INF/dal/model/appData-manifest.xml,
<webResources>
]]>
</manifest>
<resource>
</configuration>
<directory>
src/main/resources
</directory>
</execution>
<filtering>
false
</filtering>
</executions>
<targetPath>
WEB-INF/classes
</targetPath>
</plugin>
</resource>
<plugin>
<resource>
<groupId>
org.apache.maven.plugins
</groupId>
<directory>
src/main/webapp
</directory>
<artifactId>
maven-war-plugin
</artifactId>
<filtering>
false
</filtering>
<version>
2.1.1
</version>
<includes>
<configuration>
<include>
WEB-INF/web.xml
</include>
<warName>
broker-service-${env}-${version}
</warName>
</includes>
<webResources>
</resource>
<resource>
</webResources>
<directory>
src/main/resources
</directory>
</configuration>
<filtering>
false
</filtering>
</plugin>
<targetPath>
WEB-INF/classes
</targetPath>
<plugin>
</resource>
<groupId>
org.mortbay.jetty
</groupId>
<resource>
<artifactId>
maven-jetty-plugin
</artifactId>
<directory>
src/main/webapp
</directory>
<version>
6.1.26
</version>
<filtering>
false
</filtering>
<configuration>
<includes>
<scanIntervalSeconds>
10
</scanIntervalSeconds>
<include>
WEB-INF/web.xml
</include>
<contextPath>
/cat
</contextPath>
</includes>
<defaultsDescriptor>
src/main/webapp/WEB-INF/web.xml
</defaultsDescriptor>
</resource>
<connectors>
</webResources>
<connector
implementation=
"org.mortbay.jetty.nio.SelectChannelConnector"
>
</configuration>
<port>
2281
</port>
</plugin>
<maxIdleTime>
500000
</maxIdleTime>
<plugin>
</connector>
<groupId>
org.mortbay.jetty
</groupId>
</connectors>
<artifactId>
maven-jetty-plugin
</artifactId>
</configuration>
<version>
6.1.26
</version>
</plugin>
<configuration>
</plugins>
<scanIntervalSeconds>
10
</scanIntervalSeconds>
</build>
<contextPath>
/cat
</contextPath>
<properties>
<defaultsDescriptor>
src/main/webapp/WEB-INF/web.xml
</defaultsDescriptor>
<packaging>
war
</packaging>
<connectors>
<env>
alpha
</env>
<connector
implementation=
"org.mortbay.jetty.nio.SelectChannelConnector"
>
<project.build.sourceEncoding>
utf-8
</project.build.sourceEncoding>
<port>
2281
</port>
</properties>
<maxIdleTime>
500000
</maxIdleTime>
</connector>
</connectors>
</configuration>
</plugin>
</plugins>
</build>
<properties>
<packaging>
war
</packaging>
<env>
alpha
</env>
<project.build.sourceEncoding>
utf-8
</project.build.sourceEncoding>
</properties>
</project>
</project>
cat-broker/src/main/java/com/dianping/cat/broker/api/app/AppDataConsumer.java
0 → 100644
浏览文件 @
ac3a7cd4
package
com.dianping.cat.broker.api.app
;
import
java.util.concurrent.ConcurrentHashMap
;
import
org.codehaus.plexus.logging.LogEnabled
;
import
org.codehaus.plexus.logging.Logger
;
import
org.codehaus.plexus.personality.plexus.lifecycle.phase.Initializable
;
import
org.codehaus.plexus.personality.plexus.lifecycle.phase.InitializationException
;
import
org.unidal.helper.Threads
;
import
org.unidal.helper.Threads.Task
;
import
com.dianping.cat.Cat
;
import
com.dianping.cat.service.appData.entity.AppData
;
public
class
AppDataConsumer
implements
Initializable
,
LogEnabled
{
public
static
final
long
DURATION
=
5
*
60
*
1000L
;
private
AppDataQueue
m_appDataQueue
;
private
long
m_dataLoss
;
private
Logger
m_logger
;
private
ConcurrentHashMap
<
Long
,
BucketHandler
>
m_tasks
;
@Override
public
void
enableLogging
(
Logger
logger
)
{
m_logger
=
logger
;
}
public
void
enqueue
(
AppData
appData
)
{
m_appDataQueue
.
offer
(
appData
);
}
public
long
getDataLoss
()
{
return
m_dataLoss
;
}
@Override
public
void
initialize
()
throws
InitializationException
{
m_dataLoss
=
0
;
m_appDataQueue
=
new
AppDataQueue
();
m_tasks
=
new
ConcurrentHashMap
<
Long
,
BucketHandler
>();
AppDataDispatcherThread
appDataDispatcherThread
=
new
AppDataDispatcherThread
();
BucketThreadController
bucketThreadController
=
new
BucketThreadController
();
Threads
.
forGroup
(
"Cat"
).
start
(
appDataDispatcherThread
);
Threads
.
forGroup
(
"Cat"
).
start
(
bucketThreadController
);
}
public
void
setDataLoss
(
long
dataLoss
)
{
m_dataLoss
=
dataLoss
;
}
private
class
AppDataDispatcherThread
implements
Task
{
private
static
final
String
NAME
=
"AppDataDispatcherThread"
;
@Override
public
String
getName
()
{
return
NAME
;
}
@Override
public
void
run
()
{
while
(
true
)
{
try
{
AppData
appData
=
m_appDataQueue
.
poll
();
if
(
appData
!=
null
)
{
long
timestamp
=
appData
.
getTimestamp
();
timestamp
-=
timestamp
%
DURATION
;
BucketHandler
handler
=
m_tasks
.
get
(
new
Long
(
timestamp
));
if
(
handler
==
null
)
{
m_dataLoss
++;
if
(
m_dataLoss
%
1000
==
0
)
{
m_logger
.
error
(
"error timestamp in consumer, loss:"
+
m_dataLoss
);
}
}
else
{
handler
.
enqueue
(
appData
);
}
}
}
catch
(
Exception
e
)
{
Cat
.
logError
(
e
);
}
}
}
@Override
public
void
shutdown
()
{
}
}
private
class
BucketThreadController
implements
Task
{
private
void
closeLastTask
(
long
currentDuration
)
{
Long
last
=
new
Long
(
currentDuration
-
DURATION
);
BucketHandler
lastBucketHandler
=
m_tasks
.
get
(
last
);
if
(
lastBucketHandler
!=
null
)
{
lastBucketHandler
.
shutdown
();
}
}
@Override
public
String
getName
()
{
return
"BucketThreadController"
;
}
private
void
removeLastLastTask
(
long
currentDuration
)
{
Long
lastLast
=
new
Long
(
currentDuration
-
2
*
DURATION
);
m_tasks
.
remove
(
lastLast
);
}
@Override
public
void
run
()
{
while
(
true
)
{
long
curTime
=
System
.
currentTimeMillis
();
try
{
long
currentDuration
=
curTime
-
curTime
%
DURATION
;
removeLastLastTask
(
currentDuration
);
closeLastTask
(
currentDuration
);
startCurrentTask
(
currentDuration
);
startNextTask
(
currentDuration
);
}
catch
(
Exception
e
)
{
Cat
.
logError
(
e
);
}
long
elapsedTime
=
System
.
currentTimeMillis
()
-
curTime
;
try
{
Thread
.
sleep
(
DURATION
-
elapsedTime
);
}
catch
(
InterruptedException
e
)
{
}
}
}
@Override
public
void
shutdown
()
{
}
private
void
startCurrentTask
(
long
currentDuration
)
{
Long
cur
=
new
Long
(
currentDuration
);
if
(
m_tasks
.
get
(
cur
)
!=
null
)
{
BucketHandler
curBucketHandler
=
new
BucketHandler
(
cur
);
Threads
.
forGroup
(
"Cat"
).
start
(
curBucketHandler
);
m_tasks
.
put
(
cur
,
curBucketHandler
);
}
}
private
void
startNextTask
(
long
currentDuration
)
{
Long
next
=
new
Long
(
currentDuration
+
DURATION
);
if
(
m_tasks
.
get
(
next
)
!=
null
)
{
BucketHandler
nextBucketHandler
=
new
BucketHandler
(
next
);
Threads
.
forGroup
(
"Cat"
).
start
(
nextBucketHandler
);
m_tasks
.
put
(
next
,
nextBucketHandler
);
}
}
}
}
cat-broker/src/main/java/com/dianping/cat/broker/api/app/AppDataQueue.java
0 → 100644
浏览文件 @
ac3a7cd4
package
com.dianping.cat.broker.api.app
;
import
java.util.concurrent.BlockingQueue
;
import
java.util.concurrent.LinkedBlockingQueue
;
import
java.util.concurrent.TimeUnit
;
import
com.dianping.cat.service.appData.entity.AppData
;
public
class
AppDataQueue
{
private
BlockingQueue
<
AppData
>
m_dataQueue
=
new
LinkedBlockingQueue
<
AppData
>();
public
boolean
offer
(
AppData
appData
)
{
return
m_dataQueue
.
offer
(
appData
);
}
public
AppData
poll
()
{
AppData
appData
;
try
{
appData
=
m_dataQueue
.
poll
(
5
,
TimeUnit
.
MICROSECONDS
);
}
catch
(
InterruptedException
e
)
{
return
null
;
}
return
appData
;
}
}
cat-broker/src/main/java/com/dianping/cat/broker/api/app/BucketHandler.java
0 → 100644
浏览文件 @
ac3a7cd4
package
com.dianping.cat.broker.api.app
;
import
java.io.BufferedWriter
;
import
java.io.FileWriter
;
import
java.io.IOException
;
import
java.text.SimpleDateFormat
;
import
java.util.Date
;
import
java.util.HashMap
;
import
java.util.LinkedHashMap
;
import
java.util.Map.Entry
;
import
org.codehaus.plexus.logging.LogEnabled
;
import
org.codehaus.plexus.logging.Logger
;
import
org.unidal.helper.Threads.Task
;
import
com.dianping.cat.service.appData.entity.AppData
;
public
class
BucketHandler
implements
Task
,
LogEnabled
{
private
Logger
m_logger
;
private
AppDataQueue
m_appDataQueue
;
private
HashMap
<
Integer
,
HashMap
<
String
,
AppData
>>
m_mergedData
;
private
long
m_startTime
;
private
boolean
m_isActive
=
true
;
private
static
final
String
FILEDIRECTORY
=
"/data/appdatas/cat/appdata/"
;
public
void
setActive
(
boolean
isActive
)
{
m_isActive
=
isActive
;
}
public
BucketHandler
(
long
startTime
)
{
m_startTime
=
startTime
;
m_appDataQueue
=
new
AppDataQueue
();
m_mergedData
=
new
LinkedHashMap
<
Integer
,
HashMap
<
String
,
AppData
>>();
}
@Override
public
void
enableLogging
(
Logger
logger
)
{
m_logger
=
logger
;
}
@Override
public
void
run
()
{
while
(
m_isActive
)
{
AppData
appData
=
m_appDataQueue
.
poll
();
if
(
appData
!=
null
)
{
processEntity
(
appData
);
}
}
while
(
true
)
{
AppData
appData
=
m_appDataQueue
.
poll
();
if
(
appData
!=
null
)
{
processEntity
(
appData
);
}
else
{
break
;
}
}
end
();
}
private
void
processEntity
(
AppData
appData
)
{
Integer
command
=
appData
.
getCommand
();
String
key
=
m_startTime
+
":"
+
appData
.
getCity
()
+
":"
+
appData
.
getOperator
()
+
":"
+
appData
.
getChannel
()
+
":"
+
appData
.
getVersion
()
+
":"
+
appData
.
getNetwork
()
+
":"
+
appData
.
getCode
()
+
":"
+
appData
.
getPlatform
();
HashMap
<
String
,
AppData
>
secondMap
=
m_mergedData
.
get
(
command
);
if
(
secondMap
==
null
)
{
secondMap
=
new
LinkedHashMap
<
String
,
AppData
>();
secondMap
.
put
(
key
,
appData
);
m_mergedData
.
put
(
command
,
secondMap
);
}
else
{
AppData
mergedAppData
=
secondMap
.
get
(
key
);
if
(
mergedAppData
==
null
)
{
secondMap
.
put
(
key
,
appData
);
}
else
{
mergedAppData
.
setCount
(
mergedAppData
.
getCount
()
+
1
);
mergedAppData
.
setRequestByte
(
mergedAppData
.
getRequestByte
()
+
appData
.
getRequestByte
());
mergedAppData
.
setResponseByte
(
mergedAppData
.
getResponseByte
()
+
appData
.
getResponseByte
());
mergedAppData
.
setResponseTime
(
mergedAppData
.
getResponseTime
()
+
appData
.
getResponseTime
());
}
}
}
private
void
end
()
{
for
(
Entry
<
Integer
,
HashMap
<
String
,
AppData
>>
outerEntry
:
m_mergedData
.
entrySet
())
{
for
(
Entry
<
String
,
AppData
>
entry
:
outerEntry
.
getValue
().
entrySet
())
{
AppData
appData
=
entry
.
getValue
();
if
(
toHbase
(
appData
)
==
false
)
{
saveToFile
(
appData
);
}
}
}
}
@Override
public
String
getName
()
{
return
"BucketHandler"
;
}
@Override
public
void
shutdown
()
{
m_isActive
=
false
;
}
public
void
enqueue
(
AppData
appData
)
{
m_appDataQueue
.
offer
(
appData
);
}
private
boolean
toHbase
(
AppData
appData
)
{
return
false
;
}
private
void
saveToFile
(
AppData
appData
)
{
Date
date
=
new
Date
();
SimpleDateFormat
formater
=
new
SimpleDateFormat
(
"yyyy-MM-dd"
);
String
dateStr
=
formater
.
format
(
date
);
String
filePath
=
FILEDIRECTORY
+
dateStr
;
try
{
BufferedWriter
writer
=
new
BufferedWriter
(
new
FileWriter
(
filePath
));
String
content
=
appData
.
getTimestamp
()
+
"\t"
+
appData
.
getCity
()
+
"\t"
+
appData
.
getOperator
()
+
"\t"
+
appData
.
getNetwork
()
+
"\t"
+
appData
.
getVersion
()
+
"\t"
+
appData
.
getChannel
()
+
"\t"
+
appData
.
getCommand
()
+
"\t"
+
appData
.
getCode
()
+
"\t"
+
appData
.
getPlatform
()
+
"\t"
+
appData
.
getRequestByte
()
+
"\t"
+
appData
.
getResponseByte
()
+
"\t"
+
appData
.
getResponseTime
()
+
"\n"
;
writer
.
append
(
content
);
writer
.
close
();
}
catch
(
IOException
e
)
{
m_logger
.
error
(
"save appdata to file "
+
filePath
+
" failed. "
+
e
.
getMessage
());
}
}
}
cat-broker/src/main/java/com/dianping/cat/broker/api/page/IpService.java
浏览文件 @
ac3a7cd4
...
@@ -13,14 +13,14 @@ import org.codehaus.plexus.personality.plexus.lifecycle.phase.InitializationExce
...
@@ -13,14 +13,14 @@ import org.codehaus.plexus.personality.plexus.lifecycle.phase.InitializationExce
import
com.dianping.cat.Cat
;
import
com.dianping.cat.Cat
;
public
class
IpService
implements
Initializable
{
public
class
IpService
implements
Initializable
{
private
Map
<
Integer
,
Area
>
m_areas
;
private
Map
<
Integer
,
Corporation
>
m_corps
;
private
int
[]
m_areaIds
;
private
int
[]
m_areaIds
;
private
Map
<
Integer
,
Area
>
m_areas
;
private
int
[]
m_corpIds
;
private
int
[]
m_corpIds
;
private
Map
<
Integer
,
Corporation
>
m_corps
;
private
long
[]
m_ends
;
private
long
[]
m_ends
;
private
long
[]
m_starts
;
private
long
[]
m_starts
;
...
@@ -35,10 +35,12 @@ public class IpService implements Initializable {
...
@@ -35,10 +35,12 @@ public class IpService implements Initializable {
Area
area
=
m_areas
.
get
(
m_areaIds
[
mid
]);
Area
area
=
m_areas
.
get
(
m_areaIds
[
mid
]);
if
(
area
!=
null
)
{
if
(
area
!=
null
)
{
ipInfo
.
setAreaId
(
area
.
getAreaId
());
ipInfo
.
setNation
(
area
.
getNation
());
ipInfo
.
setNation
(
area
.
getNation
());
ipInfo
.
setProvince
(
area
.
getProvince
());
ipInfo
.
setProvince
(
area
.
getProvince
());
ipInfo
.
setCity
(
area
.
getCity
());
ipInfo
.
setCity
(
area
.
getCity
());
}
else
{
}
else
{
ipInfo
.
setAreaId
(
0
);
ipInfo
.
setNation
(
"未知"
);
ipInfo
.
setNation
(
"未知"
);
ipInfo
.
setProvince
(
"未知"
);
ipInfo
.
setProvince
(
"未知"
);
ipInfo
.
setCity
(
"未知"
);
ipInfo
.
setCity
(
"未知"
);
...
@@ -47,8 +49,10 @@ public class IpService implements Initializable {
...
@@ -47,8 +49,10 @@ public class IpService implements Initializable {
Corporation
corp
=
m_corps
.
get
(
m_corpIds
[
mid
]);
Corporation
corp
=
m_corps
.
get
(
m_corpIds
[
mid
]);
if
(
corp
!=
null
)
{
if
(
corp
!=
null
)
{
ipInfo
.
setCorpId
(
corp
.
getCorporationId
());
ipInfo
.
setChannel
(
corp
.
getName
());
ipInfo
.
setChannel
(
corp
.
getName
());
}
else
{
}
else
{
ipInfo
.
setCorpId
(
0
);
ipInfo
.
setChannel
(
"其他"
);
ipInfo
.
setChannel
(
"其他"
);
}
}
return
ipInfo
;
return
ipInfo
;
...
@@ -176,12 +180,12 @@ public class IpService implements Initializable {
...
@@ -176,12 +180,12 @@ public class IpService implements Initializable {
public
static
class
Area
{
public
static
class
Area
{
private
Integer
m_areaId
;
private
Integer
m_areaId
;
private
String
m_city
;
private
String
m_nation
;
private
String
m_nation
;
private
String
m_province
;
private
String
m_province
;
private
String
m_city
;
public
Integer
getAreaId
()
{
public
Integer
getAreaId
()
{
return
m_areaId
;
return
m_areaId
;
}
}
...
@@ -240,13 +244,21 @@ public class IpService implements Initializable {
...
@@ -240,13 +244,21 @@ public class IpService implements Initializable {
}
}
public
static
class
IpInfo
{
public
static
class
IpInfo
{
private
String
m_nation
;
private
Integer
m_areaId
;
private
String
m_
province
;
private
String
m_
channel
;
private
String
m_city
;
private
String
m_city
;
private
String
m_channel
;
private
Integer
m_corpId
;
private
String
m_nation
;
private
String
m_province
;
public
Integer
getAreaId
()
{
return
m_areaId
;
}
public
String
getChannel
()
{
public
String
getChannel
()
{
return
m_channel
;
return
m_channel
;
...
@@ -256,6 +268,10 @@ public class IpService implements Initializable {
...
@@ -256,6 +268,10 @@ public class IpService implements Initializable {
return
m_city
;
return
m_city
;
}
}
public
Integer
getCorpId
()
{
return
m_corpId
;
}
public
String
getNation
()
{
public
String
getNation
()
{
return
m_nation
;
return
m_nation
;
}
}
...
@@ -264,6 +280,10 @@ public class IpService implements Initializable {
...
@@ -264,6 +280,10 @@ public class IpService implements Initializable {
return
m_province
;
return
m_province
;
}
}
public
void
setAreaId
(
Integer
areaId
)
{
m_areaId
=
areaId
;
}
public
void
setChannel
(
String
name
)
{
public
void
setChannel
(
String
name
)
{
m_channel
=
name
;
m_channel
=
name
;
}
}
...
@@ -272,6 +292,10 @@ public class IpService implements Initializable {
...
@@ -272,6 +292,10 @@ public class IpService implements Initializable {
m_city
=
city
;
m_city
=
city
;
}
}
public
void
setCorpId
(
Integer
corpId
)
{
m_corpId
=
corpId
;
}
public
void
setNation
(
String
nation
)
{
public
void
setNation
(
String
nation
)
{
m_nation
=
nation
;
m_nation
=
nation
;
}
}
...
...
cat-broker/src/main/java/com/dianping/cat/broker/api/page/batch/Handler.java
浏览文件 @
ac3a7cd4
...
@@ -15,21 +15,31 @@ import org.unidal.web.mvc.annotation.InboundActionMeta;
...
@@ -15,21 +15,31 @@ import org.unidal.web.mvc.annotation.InboundActionMeta;
import
org.unidal.web.mvc.annotation.OutboundActionMeta
;
import
org.unidal.web.mvc.annotation.OutboundActionMeta
;
import
org.unidal.web.mvc.annotation.PayloadMeta
;
import
org.unidal.web.mvc.annotation.PayloadMeta
;
import
com.dianping.cat.broker.api.app.AppDataConsumer
;
import
com.dianping.cat.broker.api.page.Constrants
;
import
com.dianping.cat.broker.api.page.Constrants
;
import
com.dianping.cat.broker.api.page.IpService
;
import
com.dianping.cat.broker.api.page.IpService.IpInfo
;
import
com.dianping.cat.broker.api.page.MonitorEntity
;
import
com.dianping.cat.broker.api.page.MonitorEntity
;
import
com.dianping.cat.broker.api.page.MonitorManager
;
import
com.dianping.cat.broker.api.page.MonitorManager
;
import
com.dianping.cat.broker.api.page.RequestUtils
;
import
com.dianping.cat.broker.api.page.RequestUtils
;
import
com.dianping.cat.service.appData.entity.AppData
;
public
class
Handler
implements
PageHandler
<
Context
>,
LogEnabled
{
public
class
Handler
implements
PageHandler
<
Context
>,
LogEnabled
{
@Inject
@Inject
private
MonitorManager
m_manag
er
;
private
AppDataConsumer
m_appDataConsum
er
;
@Inject
@Inject
private
RequestUtils
m_util
;
private
IpService
m_ipService
;
private
Logger
m_logger
;
private
Logger
m_logger
;
@Inject
private
MonitorManager
m_manager
;
@Inject
private
RequestUtils
m_util
;
@Override
@Override
public
void
enableLogging
(
Logger
logger
)
{
public
void
enableLogging
(
Logger
logger
)
{
m_logger
=
logger
;
m_logger
=
logger
;
...
@@ -49,7 +59,18 @@ public class Handler implements PageHandler<Context>, LogEnabled {
...
@@ -49,7 +59,18 @@ public class Handler implements PageHandler<Context>, LogEnabled {
HttpServletRequest
request
=
ctx
.
getHttpServletRequest
();
HttpServletRequest
request
=
ctx
.
getHttpServletRequest
();
HttpServletResponse
response
=
ctx
.
getHttpServletResponse
();
HttpServletResponse
response
=
ctx
.
getHttpServletResponse
();
String
userIp
=
m_util
.
getRemoteIp
(
request
);
String
userIp
=
m_util
.
getRemoteIp
(
request
);
String
version
=
payload
.
getVersion
();
if
(
version
.
equals
(
"1"
))
{
processVersion1
(
payload
,
request
,
userIp
);
}
else
if
(
version
.
equals
(
"2"
))
{
processVersion2
(
payload
,
request
,
userIp
);
}
response
.
getWriter
().
write
(
"OK"
);
}
private
void
processVersion1
(
Payload
payload
,
HttpServletRequest
request
,
String
userIp
)
{
if
(
userIp
!=
null
)
{
if
(
userIp
!=
null
)
{
try
{
try
{
String
content
=
payload
.
getContent
();
String
content
=
payload
.
getContent
();
...
@@ -88,7 +109,46 @@ public class Handler implements PageHandler<Context>, LogEnabled {
...
@@ -88,7 +109,46 @@ public class Handler implements PageHandler<Context>, LogEnabled {
}
else
{
}
else
{
m_logger
.
info
(
"unknown http request, x-forwarded-for:"
+
request
.
getHeader
(
"x-forwarded-for"
));
m_logger
.
info
(
"unknown http request, x-forwarded-for:"
+
request
.
getHeader
(
"x-forwarded-for"
));
}
}
response
.
getWriter
().
write
(
"OK"
);
}
private
void
processVersion2
(
Payload
payload
,
HttpServletRequest
request
,
String
userIp
)
{
if
(
userIp
!=
null
)
{
String
content
=
payload
.
getContent
();
String
records
[]
=
content
.
split
(
"\n"
);
IpInfo
ipInfo
=
m_ipService
.
findIpInfoByString
(
userIp
);
for
(
String
record
:
records
)
{
String
items
[]
=
record
.
split
(
"\t"
);
if
(
items
.
length
!=
10
)
{
continue
;
}
AppData
appData
=
new
AppData
();
try
{
appData
.
setTimestamp
(
Long
.
parseLong
(
items
[
0
]));
appData
.
setNetwork
(
Integer
.
parseInt
(
items
[
1
]));
appData
.
setVersion
(
Integer
.
parseInt
(
items
[
2
]));
appData
.
setChannel
(
Integer
.
parseInt
(
items
[
3
]));
appData
.
setCommand
(
Integer
.
parseInt
(
items
[
4
]));
appData
.
setCode
(
Integer
.
parseInt
(
items
[
5
]));
appData
.
setPlatform
(
Integer
.
parseInt
(
items
[
6
]));
appData
.
setRequestByte
(
Integer
.
parseInt
(
items
[
7
]));
appData
.
setResponseByte
(
Integer
.
parseInt
(
items
[
8
]));
appData
.
setResponseTime
(
Integer
.
parseInt
(
items
[
9
]));
appData
.
setCity
(
ipInfo
.
getAreaId
());
appData
.
setOperator
(
ipInfo
.
getCorpId
());
}
catch
(
Exception
e
)
{
m_logger
.
error
(
e
.
getMessage
(),
e
);
}
m_appDataConsumer
.
enqueue
(
appData
);
}
}
else
{
m_logger
.
info
(
"unknown http request, x-forwarded-for:"
+
request
.
getHeader
(
"x-forwarded-for"
));
}
}
}
private
boolean
validate
(
String
errorCode
,
String
httpStatus
)
{
private
boolean
validate
(
String
errorCode
,
String
httpStatus
)
{
...
...
cat-broker/src/main/java/com/dianping/cat/broker/build/ComponentsConfigurator.java
浏览文件 @
ac3a7cd4
...
@@ -20,7 +20,7 @@ public class ComponentsConfigurator extends AbstractResourceConfigurator {
...
@@ -20,7 +20,7 @@ public class ComponentsConfigurator extends AbstractResourceConfigurator {
all
.
add
(
C
(
IpService
.
class
));
all
.
add
(
C
(
IpService
.
class
));
all
.
add
(
C
(
RequestUtils
.
class
));
all
.
add
(
C
(
RequestUtils
.
class
));
all
.
add
(
C
(
MonitorManager
.
class
).
req
(
UrlPatternConfigManager
.
class
,
IpService
.
class
));
all
.
add
(
C
(
MonitorManager
.
class
).
req
(
UrlPatternConfigManager
.
class
,
IpService
.
class
));
// database
// database
all
.
add
(
C
(
JdbcDataSourceDescriptorManager
.
class
)
//
all
.
add
(
C
(
JdbcDataSourceDescriptorManager
.
class
)
//
.
config
(
E
(
"datasourceFile"
).
value
(
"/data/appdatas/cat/datasources.xml"
)));
.
config
(
E
(
"datasourceFile"
).
value
(
"/data/appdatas/cat/datasources.xml"
)));
...
...
cat-broker/src/main/resources/META-INF/dal/model/appData-codegen.xml
0 → 100644
浏览文件 @
ac3a7cd4
<?xml version="1.0" encoding="UTF-8"?>
<model>
<entity
name=
"appData"
root=
"true"
>
<attribute
name=
"timestamp"
value-type=
"long"
/>
<attribute
name=
"city"
value-type=
"int"
/>
<attribute
name=
"operator"
value-type=
"int"
/>
<attribute
name=
"network"
value-type=
"int"
/>
<attribute
name=
"version"
value-type=
"int"
/>
<attribute
name=
"channel"
value-type=
"int"
/>
<attribute
name=
"command"
value-type=
"int"
/>
<attribute
name=
"code"
value-type=
"int"
/>
<attribute
name=
"platform"
value-type=
"int"
/>
<attribute
name=
"count"
value-type=
"int"
/>
<attribute
name=
"requestByte"
value-type=
"int"
/>
<attribute
name=
"responseByte"
value-type=
"int"
/>
<attribute
name=
"responseTime"
value-type=
"int"
/>
</entity>
</model>
cat-broker/src/main/resources/META-INF/dal/model/appData-manifest.xml
0 → 100644
浏览文件 @
ac3a7cd4
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<file
path=
"appData-codegen.xml"
/>
<file
path=
"appData-model.xml"
/>
</manifest>
cat-broker/src/main/resources/META-INF/dal/model/appData-model.xml
0 → 100644
浏览文件 @
ac3a7cd4
<?xml version="1.0" encoding="UTF-8"?>
<model
model-package=
"com.dianping.cat.service.appData"
/>
cat-broker/src/main/resources/META-INF/plexus/components.xml
浏览文件 @
ac3a7cd4
...
@@ -138,5 +138,30 @@
...
@@ -138,5 +138,30 @@
</requirement>
</requirement>
</requirements>
</requirements>
</component>
</component>
<component>
<role>
com.dianping.cat.broker.api.page.app.Handler
</role>
<implementation>
com.dianping.cat.broker.api.page.app.Handler
</implementation>
<requirements>
<requirement>
<role>
com.dianping.cat.broker.api.page.app.JspViewer
</role>
</requirement>
<requirement>
<role>
com.dianping.cat.broker.api.page.app.appdata.AppDataDispatcher
</role>
</requirement>
</requirements>
</component>
<component>
<role>
com.dianping.cat.broker.api.page.app.JspViewer
</role>
<implementation>
com.dianping.cat.broker.api.page.app.JspViewer
</implementation>
<requirements>
<requirement>
<role>
org.unidal.web.mvc.view.model.ModelHandler
</role>
</requirement>
</requirements>
</component>
<component>
<role>
com.dianping.cat.broker.api.page.app.appdata.AppDataDispatcher
</role>
<implementation>
com.dianping.cat.broker.api.page.app.appdata.AppDataDispatcher
</implementation>
</component>
</components>
</components>
</plexus>
</plexus>
cat-broker/src/main/resources/META-INF/wizard/model/manifest.xml
0 → 100644
浏览文件 @
ac3a7cd4
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<file
path=
"wizard.xml"
/>
</manifest>
cat-broker/src/main/resources/META-INF/wizard/model/wizard.xml
0 → 100644
浏览文件 @
ac3a7cd4
<?xml version="1.0" encoding="utf-8"?>
<wizard
package=
"com.dianping.cat.service"
>
<model
package=
"com.dianping.cat.service.appData"
name=
"appData"
>
<sample-model>
app
</sample-model>
</model>
</wizard>
cat-broker/src/main/webapp/jsp/api/app.jsp
0 → 100644
浏览文件 @
ac3a7cd4
<%@ page
contentType=
"text/html; charset=utf-8"
%>
<jsp:useBean
id=
"ctx"
type=
"com.dianping.cat.broker.api.page.app.Context"
scope=
"request"
/>
<jsp:useBean
id=
"payload"
type=
"com.dianping.cat.broker.api.page.app.Payload"
scope=
"request"
/>
<jsp:useBean
id=
"model"
type=
"com.dianping.cat.broker.api.page.app.Model"
scope=
"request"
/>
View of app page under api
\ No newline at end of file
cat-home/src/main/webapp/jsp/report/cdn.jsp
浏览文件 @
ac3a7cd4
...
@@ -102,6 +102,12 @@
...
@@ -102,6 +102,12 @@
<div
class=
"report"
>
<div
class=
"report"
>
<table>
<table>
<tr>
<tr>
<th
class=
"left"
>
<select
style=
"width: 120px;"
name=
"innerorouter"
id=
""
innerorouter
""
>
<option
value=
"inner"
>
内部cdn
</option>
<option
value=
"outer"
>
外部cdn
</option>
</select>
</th>
<th
class=
"left"
>
<th
class=
"left"
>
cdn服务商
<select
style=
"width: 120px;"
name=
"cdn"
id=
"cdn"
>
cdn服务商
<select
style=
"width: 120px;"
name=
"cdn"
id=
"cdn"
>
<option
value=
"ALL"
>
ALL
</option>
<option
value=
"ALL"
>
ALL
</option>
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录