Skip to content

C
cat

梦中观雨 / cat
与 Fork 源项目一致

从无法访问的项目Fork

通知 1
Star 0
Fork 0
C
cat

体验新版 GitCode，发现更多精彩内容 >>

提交 ac3a7cd4 编写于 作者: 曾 曾伟伟
浏览文件
操作

app data collect

上级 824801a1
隐藏空白更改
内联 并排
Showing with 706 addition and 194 deletion
cat-broker/pom.xml
浏览文件 @ ac3a7cd4
<?xml version="1.0" encoding="UTF-8"?> <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <parent>
<parent> <groupId>com.dianping.cat</groupId>
<groupId>com.dianping.cat</groupId> <artifactId>parent</artifactId>
<artifactId>parent</artifactId> <version>1.0.5</version>
<version>1.0.5</version> </parent>
</parent> <modelVersion>4.0.0</modelVersion>
<modelVersion>4.0.0</modelVersion> <artifactId>broker-service</artifactId>
<artifactId>broker-service</artifactId> <name>broker-service</name>
<name>broker-service</name> <packaging>war</packaging>
<packaging>war</packaging> <dependencies>
<dependencies> <dependency>
<dependency> <groupId>org.unidal.eunit</groupId>
<groupId>org.unidal.eunit</groupId> <artifactId>EunitTestFwk</artifactId>
<artifactId>EunitTestFwk</artifactId> <version>1.2.1</version>
<version>1.2.1</version> <scope>test</scope>
<scope>test</scope> </dependency>
</dependency> <dependency>
<dependency> <groupId>org.unidal.framework</groupId>
<groupId>org.unidal.framework</groupId> <artifactId>foundation-service</artifactId>
<artifactId>foundation-service</artifactId> <version>2.0.5</version>
<version>2.0.5</version> </dependency>
</dependency> <dependency>
<dependency> <groupId>com.dianping.cat</groupId>
<groupId>com.dianping.cat</groupId> <artifactId>cat-core</artifactId>
<artifactId>cat-core</artifactId> </dependency>
</dependency> <dependency>
<dependency> <groupId>com.google.code.gson</groupId>
<groupId>com.google.code.gson</groupId> <artifactId>gson</artifactId>
<artifactId>gson</artifactId> <version>1.6</version>
<version>1.6</version> </dependency>
</dependency> <dependency>
<dependency> <groupId>org.unidal.webres</groupId>
<groupId>org.unidal.webres</groupId> <artifactId>WebResServer</artifactId>
<artifactId>WebResServer</artifactId> <version>1.2.1</version>
<version>1.2.1</version> </dependency>
</dependency> <dependency>
<dependency> <groupId>org.unidal.framework</groupId>
<groupId>org.unidal.framework</groupId> <artifactId>web-framework</artifactId>
<artifactId>web-framework</artifactId> <version>2.0.5</version>
<version>2.0.5</version> </dependency>
</dependency> <dependency>
<dependency> <groupId>javax.servlet</groupId>
<groupId>javax.servlet</groupId> <artifactId>servlet-api</artifactId>
<artifactId>servlet-api</artifactId> <scope>provided</scope>
<scope>provided</scope> </dependency>
</dependency> <dependency>
<dependency> <groupId>javax.servlet</groupId>
<groupId>javax.servlet</groupId> <artifactId>jstl</artifactId>
<artifactId>jstl</artifactId> <version>1.2</version>
<version>1.2</version> </dependency>
</dependency> <dependency>
<dependency> <groupId>org.mortbay.jetty</groupId>
<groupId>org.mortbay.jetty</groupId> <artifactId>jsp-api-2.1</artifactId>
<artifactId>jsp-api-2.1</artifactId> <version>6.1.14</version>
<version>6.1.14</version> <scope>provided</scope>
<scope>provided</scope> </dependency>
</dependency> <dependency>
<dependency> <groupId>org.mortbay.jetty</groupId>
<groupId>org.mortbay.jetty</groupId> <artifactId>jsp-2.1</artifactId>
<artifactId>jsp-2.1</artifactId> <scope>provided</scope>
<scope>provided</scope> </dependency>
</dependency> <dependency>
<dependency> <groupId>mysql</groupId>
<groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId>
<artifactId>mysql-connector-java</artifactId> <scope>runtime</scope>
<scope>runtime</scope> </dependency>
</dependency> <dependency>
<dependency> <groupId>org.freemarker</groupId>
<groupId>org.freemarker</groupId> <artifactId>freemarker</artifactId>
<artifactId>freemarker</artifactId> <version>2.3.9</version>
<version>2.3.9</version> </dependency>
</dependency> <dependency>
<dependency> <groupId>junit</groupId>
<groupId>junit</groupId> <artifactId>junit</artifactId>
<artifactId>junit</artifactId> <scope>test</scope>
<scope>test</scope> </dependency>
</dependency> <dependency>
<dependency> <groupId>org.mortbay.jetty</groupId>
<groupId>org.mortbay.jetty</groupId> <artifactId>jetty</artifactId>
<artifactId>jetty</artifactId> <scope>test</scope>
<scope>test</scope> </dependency>
</dependency> <dependency>
<dependency> <groupId>org.unidal.framework</groupId>
<groupId>org.unidal.framework</groupId> <artifactId>test-framework</artifactId>
<artifactId>test-framework</artifactId> <version>2.0.4</version>
<version>2.0.4</version> <scope>test</scope>
<scope>test</scope> </dependency>
</dependency> <dependency>
<dependency> <groupId>org.unidal.framework</groupId>
<groupId>org.unidal.framework</groupId> <artifactId>dal-jdbc</artifactId>
<artifactId>dal-jdbc</artifactId> <version>2.0.4</version>
<version>2.0.4</version> </dependency>
</dependency> <dependency>
<dependency> <groupId>org.apache.commons</groupId>
<groupId>org.apache.commons</groupId> <artifactId>commons-email</artifactId>
<artifactId>commons-email</artifactId> <version>1.1</version>
<version>1.1</version> </dependency>
</dependency> <dependency>
<dependency> <groupId>javax.mail</groupId>
<groupId>javax.mail</groupId> <artifactId>mail</artifactId>
<artifactId>mail</artifactId> <version>1.4.4</version>
<version>1.4.4</version> </dependency>
</dependency> <dependency>
<dependency> <groupId>com.google.code.javaparser</groupId>
<groupId>com.google.code.javaparser</groupId> <artifactId>javaparser</artifactId>
<artifactId>javaparser</artifactId> <version>1.0.8</version>
<version>1.0.8</version> </dependency>
</dependency> </dependencies>
</dependencies> <build>
<build> <resources>
<resources> <resource>
<resource> <directory>${basedir}/src/main/resources</directory>
<directory>${basedir}/src/main/resources</directory> <filtering>true</filtering>
<filtering>true</filtering> </resource>
</resource> </resources>
</resources> <plugins>
<plugins> <plugin>
<plugin> <groupId>org.unidal.maven.plugins</groupId>
<groupId>org.unidal.maven.plugins</groupId> <artifactId>codegen-maven-plugin</artifactId>
<artifactId>codegen-maven-plugin</artifactId> <version>2.0.9</version>
<version>2.0.9</version> <executions>
<executions> <execution>
<execution> <id>generate plexus component descriptor</id>
<id>generate plexus component descriptor</id> <phase>process-classes</phase>
<phase>process-classes</phase> <goals>
<goals> <goal>plexus</goal>
<goal>plexus</goal> </goals>
</goals> <configuration>
<configuration> <className>com.dianping.cat.broker.build.ComponentsConfigurator</className>
<className>com.dianping.cat.broker.build.ComponentsConfigurator</className> <env>dev</env>
<env>dev</env> </configuration>
</configuration> </execution>
</execution> <execution>
</executions> <id>generate dal model files</id>
</plugin> <phase>generate-sources</phase>
<plugin> <goals>
<groupId>org.apache.maven.plugins</groupId> <goal>dal-model</goal>
<artifactId>maven-war-plugin</artifactId> </goals>
<version>2.1.1</version> <configuration>
<configuration> <manifest><![CDATA[
<warName>broker-service-${env}-${version}</warName> ${basedir}/src/main/resources/META-INF/dal/model/appData-manifest.xml,
<webResources> ]]></manifest>
<resource> </configuration>
<directory>src/main/resources</directory> </execution>
<filtering>false</filtering> </executions>
<targetPath>WEB-INF/classes</targetPath> </plugin>
</resource> <plugin>
<resource> <groupId>org.apache.maven.plugins</groupId>
<directory>src/main/webapp</directory> <artifactId>maven-war-plugin</artifactId>
<filtering>false</filtering> <version>2.1.1</version>
<includes> <configuration>
<include>WEB-INF/web.xml</include> <warName>broker-service-${env}-${version}</warName>
</includes> <webResources>
</resource> <resource>
</webResources> <directory>src/main/resources</directory>
</configuration> <filtering>false</filtering>
</plugin> <targetPath>WEB-INF/classes</targetPath>
<plugin> </resource>
<groupId>org.mortbay.jetty</groupId> <resource>
<artifactId>maven-jetty-plugin</artifactId> <directory>src/main/webapp</directory>
<version>6.1.26</version> <filtering>false</filtering>
<configuration> <includes>
<scanIntervalSeconds>10</scanIntervalSeconds> <include>WEB-INF/web.xml</include>
<contextPath>/cat</contextPath> </includes>
<defaultsDescriptor>src/main/webapp/WEB-INF/web.xml</defaultsDescriptor> </resource>
<connectors> </webResources>
<connector implementation="org.mortbay.jetty.nio.SelectChannelConnector"> </configuration>
<port>2281</port> </plugin>
<maxIdleTime>500000</maxIdleTime> <plugin>
</connector> <groupId>org.mortbay.jetty</groupId>
</connectors> <artifactId>maven-jetty-plugin</artifactId>
</configuration> <version>6.1.26</version>
</plugin> <configuration>
</plugins> <scanIntervalSeconds>10</scanIntervalSeconds>
</build> <contextPath>/cat</contextPath>
<properties> <defaultsDescriptor>src/main/webapp/WEB-INF/web.xml</defaultsDescriptor>
<packaging>war</packaging> <connectors>
<env>alpha</env> <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
<project.build.sourceEncoding>utf-8</project.build.sourceEncoding> <port>2281</port>
</properties> <maxIdleTime>500000</maxIdleTime>
</connector>
</connectors>
</configuration>
</plugin>
</plugins>
</build>
<properties>
<packaging>war</packaging>
<env>alpha</env>
<project.build.sourceEncoding>utf-8</project.build.sourceEncoding>
</properties>
</project> </project>
cat-broker/src/main/java/com/dianping/cat/broker/api/app/AppDataConsumer.java 0 → 100644
浏览文件 @ ac3a7cd4
package com.dianping.cat.broker.api.app;
import java.util.concurrent.ConcurrentHashMap;
import org.codehaus.plexus.logging.LogEnabled;
import org.codehaus.plexus.logging.Logger;
import org.codehaus.plexus.personality.plexus.lifecycle.phase.Initializable;
import org.codehaus.plexus.personality.plexus.lifecycle.phase.InitializationException;
import org.unidal.helper.Threads;
import org.unidal.helper.Threads.Task;
import com.dianping.cat.Cat;
import com.dianping.cat.service.appData.entity.AppData;
public class AppDataConsumer implements Initializable, LogEnabled {
public static final long DURATION = 5 * 60 * 1000L;
private AppDataQueue m_appDataQueue;
private long m_dataLoss;
private Logger m_logger;
private ConcurrentHashMap<Long, BucketHandler> m_tasks;
@Override
public void enableLogging(Logger logger) {
m_logger = logger;
}
public void enqueue(AppData appData) {
m_appDataQueue.offer(appData);
}
public long getDataLoss() {
return m_dataLoss;
}
@Override
public void initialize() throws InitializationException {
m_dataLoss = 0;
m_appDataQueue = new AppDataQueue();
m_tasks = new ConcurrentHashMap<Long, BucketHandler>();
AppDataDispatcherThread appDataDispatcherThread = new AppDataDispatcherThread();
BucketThreadController bucketThreadController = new BucketThreadController();
Threads.forGroup("Cat").start(appDataDispatcherThread);
Threads.forGroup("Cat").start(bucketThreadController);
}
public void setDataLoss(long dataLoss) {
m_dataLoss = dataLoss;
}
private class AppDataDispatcherThread implements Task {
private static final String NAME = "AppDataDispatcherThread";
@Override
public String getName() {
return NAME;
}
@Override
public void run() {
while (true) {
try {
AppData appData = m_appDataQueue.poll();
if (appData != null) {
long timestamp = appData.getTimestamp();
timestamp -= timestamp % DURATION;
BucketHandler handler = m_tasks.get(new Long(timestamp));
if (handler == null) {
m_dataLoss++;
if (m_dataLoss % 1000 == 0) {
m_logger.error("error timestamp in consumer, loss:" + m_dataLoss);
}
} else {
handler.enqueue(appData);
}
}
} catch (Exception e) {
Cat.logError(e);
}
}
}
@Override
public void shutdown() {
}
}
private class BucketThreadController implements Task {
private void closeLastTask(long currentDuration) {
Long last = new Long(currentDuration - DURATION);
BucketHandler lastBucketHandler = m_tasks.get(last);
if (lastBucketHandler != null) {
lastBucketHandler.shutdown();
}
}
@Override
public String getName() {
return "BucketThreadController";
}
private void removeLastLastTask(long currentDuration) {
Long lastLast = new Long(currentDuration - 2 * DURATION);
m_tasks.remove(lastLast);
}
@Override
public void run() {
while (true) {
long curTime = System.currentTimeMillis();
try {
long currentDuration = curTime - curTime % DURATION;
removeLastLastTask(currentDuration);
closeLastTask(currentDuration);
startCurrentTask(currentDuration);
startNextTask(currentDuration);
} catch (Exception e) {
Cat.logError(e);
}
long elapsedTime = System.currentTimeMillis() - curTime;
try {
Thread.sleep(DURATION - elapsedTime);
} catch (InterruptedException e) {
}
}
}
@Override
public void shutdown() {
}
private void startCurrentTask(long currentDuration) {
Long cur = new Long(currentDuration);
if (m_tasks.get(cur) != null) {
BucketHandler curBucketHandler = new BucketHandler(cur);
Threads.forGroup("Cat").start(curBucketHandler);
m_tasks.put(cur, curBucketHandler);
}
}
private void startNextTask(long currentDuration) {
Long next = new Long(currentDuration + DURATION);
if (m_tasks.get(next) != null) {
BucketHandler nextBucketHandler = new BucketHandler(next);
Threads.forGroup("Cat").start(nextBucketHandler);
m_tasks.put(next, nextBucketHandler);
}
}
}
}
cat-broker/src/main/java/com/dianping/cat/broker/api/app/AppDataQueue.java 0 → 100644
浏览文件 @ ac3a7cd4
package com.dianping.cat.broker.api.app;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.TimeUnit;
import com.dianping.cat.service.appData.entity.AppData;
public class AppDataQueue {
private BlockingQueue<AppData> m_dataQueue = new LinkedBlockingQueue<AppData>();
public boolean offer(AppData appData) {
return m_dataQueue.offer(appData);
}
public AppData poll() {
AppData appData;
try {
appData = m_dataQueue.poll(5, TimeUnit.MICROSECONDS);
} catch (InterruptedException e) {
return null;
}
return appData;
}
}
cat-broker/src/main/java/com/dianping/cat/broker/api/app/BucketHandler.java 0 → 100644
浏览文件 @ ac3a7cd4
package com.dianping.cat.broker.api.app;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.IOException;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.Map.Entry;
import org.codehaus.plexus.logging.LogEnabled;
import org.codehaus.plexus.logging.Logger;
import org.unidal.helper.Threads.Task;
import com.dianping.cat.service.appData.entity.AppData;
public class BucketHandler implements Task, LogEnabled {
private Logger m_logger;
private AppDataQueue m_appDataQueue;
private HashMap<Integer, HashMap<String, AppData>> m_mergedData;
private long m_startTime;
private boolean m_isActive = true;
private static final String FILEDIRECTORY = "/data/appdatas/cat/appdata/";
public void setActive(boolean isActive) {
m_isActive = isActive;
}
public BucketHandler(long startTime) {
m_startTime = startTime;
m_appDataQueue = new AppDataQueue();
m_mergedData = new LinkedHashMap<Integer, HashMap<String, AppData>>();
}
@Override
public void enableLogging(Logger logger) {
m_logger = logger;
}
@Override
public void run() {
while (m_isActive) {
AppData appData = m_appDataQueue.poll();
if (appData != null) {
processEntity(appData);
}
}
while (true) {
AppData appData = m_appDataQueue.poll();
if (appData != null) {
processEntity(appData);
} else {
break;
}
}
end();
}
private void processEntity(AppData appData) {
Integer command = appData.getCommand();
String key = m_startTime + ":" + appData.getCity() + ":" + appData.getOperator() + ":" + appData.getChannel()
+ ":" + appData.getVersion() + ":" + appData.getNetwork() + ":" + appData.getCode() + ":"
+ appData.getPlatform();
HashMap<String, AppData> secondMap = m_mergedData.get(command);
if (secondMap == null) {
secondMap = new LinkedHashMap<String, AppData>();
secondMap.put(key, appData);
m_mergedData.put(command, secondMap);
} else {
AppData mergedAppData = secondMap.get(key);
if (mergedAppData == null) {
secondMap.put(key, appData);
} else {
mergedAppData.setCount(mergedAppData.getCount() + 1);
mergedAppData.setRequestByte(mergedAppData.getRequestByte() + appData.getRequestByte());
mergedAppData.setResponseByte(mergedAppData.getResponseByte() + appData.getResponseByte());
mergedAppData.setResponseTime(mergedAppData.getResponseTime() + appData.getResponseTime());
}
}
}
private void end() {
for (Entry<Integer, HashMap<String, AppData>> outerEntry : m_mergedData.entrySet()) {
for (Entry<String, AppData> entry : outerEntry.getValue().entrySet()) {
AppData appData = entry.getValue();
if (toHbase(appData) == false) {
saveToFile(appData);
}
}
}
}
@Override
public String getName() {
return "BucketHandler";
}
@Override
public void shutdown() {
m_isActive = false;
}
public void enqueue(AppData appData) {
m_appDataQueue.offer(appData);
}
private boolean toHbase(AppData appData) {
return false;
}
private void saveToFile(AppData appData) {
Date date = new Date();
SimpleDateFormat formater = new SimpleDateFormat("yyyy-MM-dd");
String dateStr = formater.format(date);
String filePath = FILEDIRECTORY + dateStr;
try {
BufferedWriter writer = new BufferedWriter(new FileWriter(filePath));
String content = appData.getTimestamp() + "\t" + appData.getCity() + "\t" + appData.getOperator() + "\t"
+ appData.getNetwork() + "\t" + appData.getVersion() + "\t" + appData.getChannel() + "\t"
+ appData.getCommand() + "\t" + appData.getCode() + "\t" + appData.getPlatform() + "\t"
+ appData.getRequestByte() + "\t" + appData.getResponseByte() + "\t" + appData.getResponseTime() + "\n";
writer.append(content);
writer.close();
} catch (IOException e) {
m_logger.error("save appdata to file " + filePath + " failed. " + e.getMessage());
}
}
}
cat-broker/src/main/java/com/dianping/cat/broker/api/page/IpService.java
浏览文件 @ ac3a7cd4
...@@ -13,14 +13,14 @@ import org.codehaus.plexus.personality.plexus.lifecycle.phase.InitializationExce ...@@ -13,14 +13,14 @@ import org.codehaus.plexus.personality.plexus.lifecycle.phase.InitializationExce
import com.dianping.cat.Cat; import com.dianping.cat.Cat;
public class IpService implements Initializable { public class IpService implements Initializable {
private Map<Integer, Area> m_areas;
private Map<Integer, Corporation> m_corps;
private int[] m_areaIds; private int[] m_areaIds;
private Map<Integer, Area> m_areas;
private int[] m_corpIds; private int[] m_corpIds;
private Map<Integer, Corporation> m_corps;
private long[] m_ends; private long[] m_ends;
private long[] m_starts; private long[] m_starts;
...@@ -35,10 +35,12 @@ public class IpService implements Initializable { ...@@ -35,10 +35,12 @@ public class IpService implements Initializable {
Area area = m_areas.get(m_areaIds[mid]); Area area = m_areas.get(m_areaIds[mid]);
if (area != null) { if (area != null) {
ipInfo.setAreaId(area.getAreaId());
ipInfo.setNation(area.getNation()); ipInfo.setNation(area.getNation());
ipInfo.setProvince(area.getProvince()); ipInfo.setProvince(area.getProvince());
ipInfo.setCity(area.getCity()); ipInfo.setCity(area.getCity());
} else { } else {
ipInfo.setAreaId(0);
ipInfo.setNation("未知"); ipInfo.setNation("未知");
ipInfo.setProvince("未知"); ipInfo.setProvince("未知");
ipInfo.setCity("未知"); ipInfo.setCity("未知");
...@@ -47,8 +49,10 @@ public class IpService implements Initializable { ...@@ -47,8 +49,10 @@ public class IpService implements Initializable {
Corporation corp = m_corps.get(m_corpIds[mid]); Corporation corp = m_corps.get(m_corpIds[mid]);
if (corp != null) { if (corp != null) {
ipInfo.setCorpId(corp.getCorporationId());
ipInfo.setChannel(corp.getName()); ipInfo.setChannel(corp.getName());
} else { } else {
ipInfo.setCorpId(0);
ipInfo.setChannel("其他"); ipInfo.setChannel("其他");
} }
return ipInfo; return ipInfo;
...@@ -176,12 +180,12 @@ public class IpService implements Initializable { ...@@ -176,12 +180,12 @@ public class IpService implements Initializable {
public static class Area { public static class Area {
private Integer m_areaId; private Integer m_areaId;
private String m_city;
private String m_nation; private String m_nation;
private String m_province; private String m_province;
private String m_city;
public Integer getAreaId() { public Integer getAreaId() {
return m_areaId; return m_areaId;
} }
...@@ -240,13 +244,21 @@ public class IpService implements Initializable { ...@@ -240,13 +244,21 @@ public class IpService implements Initializable {
} }
public static class IpInfo { public static class IpInfo {
private String m_nation; private Integer m_areaId;
private String m_province; private String m_channel;
private String m_city; private String m_city;
private String m_channel; private Integer m_corpId;
private String m_nation;
private String m_province;
public Integer getAreaId() {
return m_areaId;
}
public String getChannel() { public String getChannel() {
return m_channel; return m_channel;
...@@ -256,6 +268,10 @@ public class IpService implements Initializable { ...@@ -256,6 +268,10 @@ public class IpService implements Initializable {
return m_city; return m_city;
} }
public Integer getCorpId() {
return m_corpId;
}
public String getNation() { public String getNation() {
return m_nation; return m_nation;
} }
...@@ -264,6 +280,10 @@ public class IpService implements Initializable { ...@@ -264,6 +280,10 @@ public class IpService implements Initializable {
return m_province; return m_province;
} }
public void setAreaId(Integer areaId) {
m_areaId = areaId;
}
public void setChannel(String name) { public void setChannel(String name) {
m_channel = name; m_channel = name;
} }
...@@ -272,6 +292,10 @@ public class IpService implements Initializable { ...@@ -272,6 +292,10 @@ public class IpService implements Initializable {
m_city = city; m_city = city;
} }
public void setCorpId(Integer corpId) {
m_corpId = corpId;
}
public void setNation(String nation) { public void setNation(String nation) {
m_nation = nation; m_nation = nation;
} }
......
cat-broker/src/main/java/com/dianping/cat/broker/api/page/batch/Handler.java
浏览文件 @ ac3a7cd4
...@@ -15,21 +15,31 @@ import org.unidal.web.mvc.annotation.InboundActionMeta; ...@@ -15,21 +15,31 @@ import org.unidal.web.mvc.annotation.InboundActionMeta;
import org.unidal.web.mvc.annotation.OutboundActionMeta; import org.unidal.web.mvc.annotation.OutboundActionMeta;
import org.unidal.web.mvc.annotation.PayloadMeta; import org.unidal.web.mvc.annotation.PayloadMeta;
import com.dianping.cat.broker.api.app.AppDataConsumer;
import com.dianping.cat.broker.api.page.Constrants; import com.dianping.cat.broker.api.page.Constrants;
import com.dianping.cat.broker.api.page.IpService;
import com.dianping.cat.broker.api.page.IpService.IpInfo;
import com.dianping.cat.broker.api.page.MonitorEntity; import com.dianping.cat.broker.api.page.MonitorEntity;
import com.dianping.cat.broker.api.page.MonitorManager; import com.dianping.cat.broker.api.page.MonitorManager;
import com.dianping.cat.broker.api.page.RequestUtils; import com.dianping.cat.broker.api.page.RequestUtils;
import com.dianping.cat.service.appData.entity.AppData;
public class Handler implements PageHandler<Context>, LogEnabled { public class Handler implements PageHandler<Context>, LogEnabled {
@Inject @Inject
private MonitorManager m_manager; private AppDataConsumer m_appDataConsumer;
@Inject @Inject
private RequestUtils m_util; private IpService m_ipService;
private Logger m_logger; private Logger m_logger;
@Inject
private MonitorManager m_manager;
@Inject
private RequestUtils m_util;
@Override @Override
public void enableLogging(Logger logger) { public void enableLogging(Logger logger) {
m_logger = logger; m_logger = logger;
...@@ -49,7 +59,18 @@ public class Handler implements PageHandler<Context>, LogEnabled { ...@@ -49,7 +59,18 @@ public class Handler implements PageHandler<Context>, LogEnabled {
HttpServletRequest request = ctx.getHttpServletRequest(); HttpServletRequest request = ctx.getHttpServletRequest();
HttpServletResponse response = ctx.getHttpServletResponse(); HttpServletResponse response = ctx.getHttpServletResponse();
String userIp = m_util.getRemoteIp(request); String userIp = m_util.getRemoteIp(request);
String version = payload.getVersion();
if (version.equals("1")) {
processVersion1(payload, request, userIp);
} else if (version.equals("2")) {
processVersion2(payload, request, userIp);
}
response.getWriter().write("OK");
}
private void processVersion1(Payload payload, HttpServletRequest request, String userIp) {
if (userIp != null) { if (userIp != null) {
try { try {
String content = payload.getContent(); String content = payload.getContent();
...@@ -88,7 +109,46 @@ public class Handler implements PageHandler<Context>, LogEnabled { ...@@ -88,7 +109,46 @@ public class Handler implements PageHandler<Context>, LogEnabled {
} else { } else {
m_logger.info("unknown http request, x-forwarded-for:" + request.getHeader("x-forwarded-for")); m_logger.info("unknown http request, x-forwarded-for:" + request.getHeader("x-forwarded-for"));
} }
response.getWriter().write("OK"); }
private void processVersion2(Payload payload, HttpServletRequest request, String userIp) {
if (userIp != null) {
String content = payload.getContent();
String records[] = content.split("\n");
IpInfo ipInfo = m_ipService.findIpInfoByString(userIp);
for (String record : records) {
String items[] = record.split("\t");
if (items.length != 10) {
continue;
}
AppData appData = new AppData();
try {
appData.setTimestamp(Long.parseLong(items[0]));
appData.setNetwork(Integer.parseInt(items[1]));
appData.setVersion(Integer.parseInt(items[2]));
appData.setChannel(Integer.parseInt(items[3]));
appData.setCommand(Integer.parseInt(items[4]));
appData.setCode(Integer.parseInt(items[5]));
appData.setPlatform(Integer.parseInt(items[6]));
appData.setRequestByte(Integer.parseInt(items[7]));
appData.setResponseByte(Integer.parseInt(items[8]));
appData.setResponseTime(Integer.parseInt(items[9]));
appData.setCity(ipInfo.getAreaId());
appData.setOperator(ipInfo.getCorpId());
} catch (Exception e) {
m_logger.error(e.getMessage(), e);
}
m_appDataConsumer.enqueue(appData);
}
} else {
m_logger.info("unknown http request, x-forwarded-for:" + request.getHeader("x-forwarded-for"));
}
} }
private boolean validate(String errorCode, String httpStatus) { private boolean validate(String errorCode, String httpStatus) {
......
cat-broker/src/main/java/com/dianping/cat/broker/build/ComponentsConfigurator.java
浏览文件 @ ac3a7cd4
...@@ -20,7 +20,7 @@ public class ComponentsConfigurator extends AbstractResourceConfigurator { ...@@ -20,7 +20,7 @@ public class ComponentsConfigurator extends AbstractResourceConfigurator {
all.add(C(IpService.class)); all.add(C(IpService.class));
all.add(C(RequestUtils.class)); all.add(C(RequestUtils.class));
all.add(C(MonitorManager.class).req(UrlPatternConfigManager.class, IpService.class)); all.add(C(MonitorManager.class).req(UrlPatternConfigManager.class, IpService.class));
// database // database
all.add(C(JdbcDataSourceDescriptorManager.class) // all.add(C(JdbcDataSourceDescriptorManager.class) //
.config(E("datasourceFile").value("/data/appdatas/cat/datasources.xml"))); .config(E("datasourceFile").value("/data/appdatas/cat/datasources.xml")));
......
cat-broker/src/main/resources/META-INF/dal/model/appData-codegen.xml 0 → 100644
浏览文件 @ ac3a7cd4
<?xml version="1.0" encoding="UTF-8"?>
<model>
<entity name="appData" root="true">
<attribute name="timestamp" value-type="long" />
<attribute name="city" value-type="int" />
<attribute name="operator" value-type="int" />
<attribute name="network" value-type="int" />
<attribute name="version" value-type="int" />
<attribute name="channel" value-type="int" />
<attribute name="command" value-type="int" />
<attribute name="code" value-type="int" />
<attribute name="platform" value-type="int" />
<attribute name="count" value-type="int" />
<attribute name="requestByte" value-type="int" />
<attribute name="responseByte" value-type="int" />
<attribute name="responseTime" value-type="int" />
</entity>
</model>
cat-broker/src/main/resources/META-INF/dal/model/appData-manifest.xml 0 → 100644
浏览文件 @ ac3a7cd4
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<file path="appData-codegen.xml" />
<file path="appData-model.xml" />
</manifest>
cat-broker/src/main/resources/META-INF/dal/model/appData-model.xml 0 → 100644
浏览文件 @ ac3a7cd4
<?xml version="1.0" encoding="UTF-8"?>
<model model-package="com.dianping.cat.service.appData" />
cat-broker/src/main/resources/META-INF/plexus/components.xml
浏览文件 @ ac3a7cd4
...@@ -138,5 +138,30 @@ ...@@ -138,5 +138,30 @@
</requirement> </requirement>
</requirements> </requirements>
</component> </component>
<component>
<role>com.dianping.cat.broker.api.page.app.Handler</role>
<implementation>com.dianping.cat.broker.api.page.app.Handler</implementation>
<requirements>
<requirement>
<role>com.dianping.cat.broker.api.page.app.JspViewer</role>
</requirement>
<requirement>
<role>com.dianping.cat.broker.api.page.app.appdata.AppDataDispatcher</role>
</requirement>
</requirements>
</component>
<component>
<role>com.dianping.cat.broker.api.page.app.JspViewer</role>
<implementation>com.dianping.cat.broker.api.page.app.JspViewer</implementation>
<requirements>
<requirement>
<role>org.unidal.web.mvc.view.model.ModelHandler</role>
</requirement>
</requirements>
</component>
<component>
<role>com.dianping.cat.broker.api.page.app.appdata.AppDataDispatcher</role>
<implementation>com.dianping.cat.broker.api.page.app.appdata.AppDataDispatcher</implementation>
</component>
</components> </components>
</plexus> </plexus>
cat-broker/src/main/resources/META-INF/wizard/model/manifest.xml 0 → 100644
浏览文件 @ ac3a7cd4
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<file path="wizard.xml" />
</manifest>
cat-broker/src/main/resources/META-INF/wizard/model/wizard.xml 0 → 100644
浏览文件 @ ac3a7cd4
<?xml version="1.0" encoding="utf-8"?>
<wizard package="com.dianping.cat.service">
<model package="com.dianping.cat.service.appData" name="appData">
<sample-model>app</sample-model>
</model>
</wizard>
cat-broker/src/main/webapp/jsp/api/app.jsp 0 → 100644
浏览文件 @ ac3a7cd4
<%@ page contentType="text/html; charset=utf-8" %>
<jsp:useBean id="ctx" type="com.dianping.cat.broker.api.page.app.Context" scope="request"/>
<jsp:useBean id="payload" type="com.dianping.cat.broker.api.page.app.Payload" scope="request"/>
<jsp:useBean id="model" type="com.dianping.cat.broker.api.page.app.Model" scope="request"/>
View of app page under api
\ No newline at end of file
cat-home/src/main/webapp/jsp/report/cdn.jsp
浏览文件 @ ac3a7cd4
...@@ -102,6 +102,12 @@ ...@@ -102,6 +102,12 @@
<div class="report"> <div class="report">
<table> <table>
<tr> <tr>
<th class="left">
<select style="width: 120px;" name="innerorouter" id=""innerorouter"">
<option value="inner">内部cdn</option>
<option value="outer">外部cdn</option>
</select>
</th>
<th class="left"> <th class="left">
cdn服务商 <select style="width: 120px;" name="cdn" id="cdn"> cdn服务商 <select style="width: 120px;" name="cdn" id="cdn">
<option value="ALL">ALL</option> <option value="ALL">ALL</option>
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或