Skip to content

P
Paddle-Lite

PaddlePaddle / Paddle-Lite

通知 338
Star 4
Fork 1
P
Paddle-Lite
提交 7110db47 编写于 作者: W wangqun
浏览文件
操作

[change]update paddleJS construct 

[change]update paddleJS construct
上级 cbeff2b5
展开全部 隐藏空白更改
内联 并排
Showing with 4068733 addition and 406 deletion
web/README_cn 2.md 0 → 100644
浏览文件 @ 7110db47
# PaddleJS Examples
百度PaddleJS使用现成的 JavaScript 模型或转换 Paddle 模型以在浏览器中运行。
## 演示
目前Web项目运行TinyYolo模型可以达到30ms以内，对于一般的实时场景已经足够应对。
### 模块化
## 浏览器覆盖面
* PC: Chrome
* Mac: Chrome
* Android: Baidu App and QQ Browser
## 构建部署
```bash
cd web # 进入根目录
npm i # 安装依赖
mkdir dist # 创建资源目录
cd dist # 进入资源目录
git clone https://github.com/DerekYangMing/Paddle-Web-Models.git # 获取模型
mv Paddle-Web-Models/separablemodel . # 移动模型到制定地点
cd .. # 返回根目录
npm run tinyYolo # 启动 tinyYolo 在线推理服务
```
## 如何预览 demo
1. 在浏览器中打开url: https://localhost:8123/
2. 点击【开始检测】按钮。
3. 将人脸对准摄像头，没有问题的话，可以正常检测到人脸。
## 效果
![image](./tinyYolo/demoshow.png)
web/examples/README_cn.md 0 → 100644
浏览文件 @ 7110db47
# PaddleJS Examples
百度PaddleJS使用现成的 JavaScript 模型或转换 Paddle 模型以在浏览器中运行。
## 演示
目前Web项目运行TinyYolo模型可以达到30ms以内，对于一般的实时场景已经足够应对。
### 模块化
## 浏览器覆盖面
* PC: Chrome
* Mac: Chrome
* Android: Baidu App and QQ Browser
## 构建部署
```bash
cd web # 进入根目录
npm i # 安装依赖
mkdir dist # 创建资源目录
cd dist # 进入资源目录
git clone https://github.com/DerekYangMing/Paddle-Web-Models.git # 获取模型
mv Paddle-Web-Models/separablemodel . # 移动模型到制定地点
cd .. # 返回根目录
npm run tinyYolo # 启动 tinyYolo 在线推理服务
```
## 如何预览 demo
1. 在浏览器中打开url: https://localhost:8123/
2. 点击【开始检测】按钮。
3. 将人脸对准摄像头，没有问题的话，可以正常检测到人脸。
## 效果
![image](./tinyYolo/demoshow.png)
web/examples/huangfan/index.es6 0 → 100644
浏览文件 @ 7110db47
import 'babel-polyfill';
import Paddle from '../../src/paddle/paddle';
import IO from '../../src/feed/imageFeed';
import Utils from '../../src/utils/utils';
// 获取map表
import Map from '../../test/data/map';
/**
* @file model demo 入口文件
* @author wangqun@baidu.com
*
*/
// 模型feed数据
const feedShape = {
'608': {
fw: 608,
fh: 608
},
'320': {
fw: 320,
fh: 320
},
'320fused': {
fw: 320,
fh: 320
},
'separate': {
fw: 320,
fh: 320
}
};
const modelType = 'separate';
const {fw, fh} = feedShape[modelType];
// 统计参数
let loaded = false;
let model = {};
window.statistic = [];
async function run(input) {
// const input = document.getElementById('mobilenet');
const io = new IO();
let feed = io.process({
input: input,
params: {
targetShape: [1, 3, fh, fw], // 目标形状 为了兼容之前的逻辑所以改个名
scale: 256, // 缩放尺寸
width: 224, height: 224, // 压缩宽高
shape: [3, 224, 224], // 预设tensor形状
mean: [0.485, 0.456, 0.406], // 预设期望
std: [0.229, 0.224, 0.225] // 预设方差
}});
console.dir(['feed', feed]);
const path = 'model/huangfan';
if (!loaded) {
const MODEL_CONFIG = {
dir: `/${path}/`, // 存放模型的文件夹
main: 'model.json', // 主文件
};
loaded = true;
const paddle = new Paddle({
urlConf: MODEL_CONFIG,
options: {
multipart: false,
dataType: 'json'
}
});
model = await paddle.load();
}
let inst = model.execute({
input: feed
});
// 其实这里应该有个fetch的执行调用或者fetch的输出
let result = await inst.read();
console.dir(['result', result]);
let maxItem = Utils.getMaxItem(result);
document.getElementById('txt').innerHTML = Map['' + maxItem.index];
console.log('识别出的结果是' + Map['' + maxItem.index]);
// console.dir(['每个op耗时', window.statistic]);
// let total = statistic.reduce((all, cur) => {
// return all + cur.runTime;
// }, 0);
// console.log('op total = ' + total);
}
var image = '';
function selectImage(file) {
if (!file.files || !file.files[0]) {
return;
}
let reader = new FileReader();
reader.onload = function (evt) {
let img = document.getElementById('image');
img.src = evt.target.result;
img.onload = function () {
run(img);
};
image = evt.target.result;
};
reader.readAsDataURL(file.files[0]);
}
// selectImage
document.getElementById('uploadImg').onchange = function () {
selectImage(this);
};
web/examples/huangfan/index.html 0 → 100644
浏览文件 @ 7110db47
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>paddle web demo</title>
<meta name="viewport" content="width=device-width,minimum-scale=1.0,maximum-scale=1.0,user-scalable=no">
</head>
<body>
<img id="image" src="https://m.baidu.com/se/static/img/iphone/logo.png" style="max-width: 100%;">
<input type="file" id="uploadImg">
<div id="txt"></div>
<script src="index.es6"></script>
</body>
</html>
\ No newline at end of file
web/examples/mnist/index.es6 0 → 100644
浏览文件 @ 7110db47
import 'babel-polyfill';
import Paddle from '../../src/paddle/paddle';
import IO from '../../src/feed/imageFeed';
/**
* @file model demo mnist 入口文件
* @author wangqun@baidu.com
*
*/
const pic = document.getElementById('pic');
const io = new IO();
let model = {};
async function run() {
let feed = io.process({
input: pic,
params: {
targetShape: [1, 3, 320, 320], // 目标形状 为了兼容之前的逻辑所以改个名
scale: 256, // 缩放尺寸
width: 224, height: 224, // 压缩宽高
shape: [3, 224, 224], // 预设tensor形状
mean: [0.485, 0.456, 0.406], // 预设期望
std: [0.229, 0.224, 0.225] // 预设方差
}});
console.dir(['feed', feed]);
const path = 'model/mnist';
const MODEL_CONFIG = {
dir: `/${path}/`, // 存放模型的文件夹
main: 'model.json', // 主文件
};
const paddle = new Paddle({
urlConf: MODEL_CONFIG,
options: {
multipart: false,
dataType: 'json'
}
});
model = await paddle.load();
let inst = model.execute({
input: feed
});
// 其实这里应该有个fetch的执行调用或者fetch的输出
let result = await inst.read();
// let inst = model.execute({input: cat});
// let res = inst.read();
console.dir(['result', result]);
// var fileDownload = require('js-file-download');
// fileDownload(res, 'result.csv');
}
run();
\ No newline at end of file
web/examples/mnist/index.html 0 → 100644
浏览文件 @ 7110db47
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>paddle web demo</title>
<meta name="viewport" content="width=device-width,minimum-scale=1.0,maximum-scale=1.0,user-scalable=no">
</head>
<body>
<div>
<img id="pic" src="data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/wAALCAAcABwBAREA/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/9oACAEBAAA/APn+vTPDHwP8TeJ9DtdXiuLCzt7kbo0uWcOU7NgKRgjkc81i+O/hvrPgW8xco1zp7ELHfIm1HYqCRjJIPUc9cHFcbSgEnABJ9BXaafH8Rrrw3NpdjBrkmjohLQLE/l7c5OOPUHgV6Fcw3um/sxXNt4hZo7qW5X7FDdLtlRfOU7QG5zgSH/dPpXhFel/Bzxj4a8H6vfzeILZy86ILe6WLzPI27i3HUZ+XkA9PQ16Pc/Hfw7pM91LaXusa20wDRxSQRQww9eAdob35DfWuNg+Ny67Dfab430SDUNLuQxjW2UK8BwcAZPPOPmyCOvPSvH6KKKK//9k=" >
</div>
<script src="index.es6"></script>
</body>
</html>
web/examples/mobileNet/index.es6 0 → 100644
浏览文件 @ 7110db47
import 'babel-polyfill';
import Paddle from '../../src/paddle/paddle';
import IO from '../../src/feed/imageFeed';
import Utils from '../../src/utils/utils';
// 获取map表
import Map from '../../test/data/map';
/**
* @file model demo 入口文件
* @author wangqun@baidu.com
*
*/
// 模型feed数据
const feedShape = {
'608': {
fw: 608,
fh: 608
},
'320': {
fw: 320,
fh: 320
},
'320fused': {
fw: 320,
fh: 320
},
'separate': {
fw: 244,
fh: 244
}
};
const modelType = 'separate';
const {fw, fh} = feedShape[modelType];
// 统计参数
let loaded = false;
let model = {};
window.statistic = [];
async function run(input) {
// const input = document.getElementById('mobilenet');
const io = new IO();
let feed = io.process({
input: input,
params: {
targetShape: [1, 3, fh, fw], // 目标形状 为了兼容之前的逻辑所以改个名
scale: 256, // 缩放尺寸
width: 224, height: 224, // 压缩宽高
shape: [3, 224, 224], // 预设tensor形状
mean: [0.485, 0.456, 0.406], // 预设期望
std: [0.229, 0.224, 0.225] // 预设方差
}});
console.log('feed', feed);
const path = 'model/mobileNet';
if (!loaded) {
const MODEL_CONFIG = {
dir: `/${path}/`, // 存放模型的文件夹
main: 'model.json', // 主文件
};
loaded = true;
const paddle = new Paddle({
urlConf: MODEL_CONFIG,
options: {
multipart: true,
dataType: 'json'
}
});
model = await paddle.load();
}
let inst = model.execute({
input: feed
});
// 其实这里应该有个fetch的执行调用或者fetch的输出
let result = await inst.read();
console.dir(['result', result]);
// let maxItem = Utils.getMaxItem(result);
// document.getElementById('txt').innerHTML = Map['' + maxItem.index];
// console.log('识别出的结果是' + Map['' + maxItem.index]);
// console.dir(['每个op耗时', window.statistic]);
// let total = statistic.reduce((all, cur) => {
// return all + cur.runTime;
// }, 0);
// console.log('op total = ' + total);
};
var image = '';
function selectImage(file) {
if (!file.files || !file.files[0]) {
return;
}
let reader = new FileReader();
reader.onload = function (evt) {
let img = document.getElementById('image');
img.src = evt.target.result;
img.onload = function() {
run(img);
};
image = evt.target.result;
}
reader.readAsDataURL(file.files[0]);
}
// selectImage
document.getElementById("uploadImg").onchange = function () {
selectImage(this);
};
web/examples/mobileNet/index.html 0 → 100644
浏览文件 @ 7110db47
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>paddle web demo</title>
<meta name="viewport" content="width=device-width,minimum-scale=1.0,maximum-scale=1.0,user-scalable=no">
</head>
<body>
<img id="image" src="https://m.baidu.com/se/static/img/iphone/logo.png" style="max-width: 100%;">
<input type="file" id="uploadImg">
<div id="txt"></div>
<script src="index.es6"></script>
</body>
</html>
\ No newline at end of file
web/examples/model/model.json 0 → 100644
浏览文件 @ 7110db47
此差异已折叠。
web/examples/tinyYolo/camera.es6 0 → 100644
浏览文件 @ 7110db47
export default class Camera {
constructor(option) {
this.video = option.videoDom;
this.videoOption = option.videoOption;
}
// 访问用户媒体设备的兼容方法
getUserMedia(constraints, success, error) {
if (navigator.mediaDevices.getUserMedia) {
// 最新的标准API
navigator.mediaDevices.getUserMedia(constraints).then(success).catch(error);
}
else if (navigator.webkitGetUserMedia) {
// webkit核心浏览器
navigator.webkitGetUserMedia(constraints, success, error);
}
else if (navigator.mozGetUserMedia) {
// firfox浏览器
navigator.mozGetUserMedia(constraints, success, error);
}
else if (navigator.getUserMedia) {
// 旧版API
navigator.getUserMedia(constraints, success, error);
}
}
success(stream) {
// 兼容webkit核心浏览器
let CompatibleURL = window.URL || window.webkitURL;
// 将视频流设置为video元素的源
// video.src = CompatibleURL.createObjectURL(stream);
this.video.srcObject = stream;
this.video.play();
}
error(error) {
console.log(`访问用户媒体设备失败${error.name}, ${error.message}`);
}
run() {
if (navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.mediaDevices.getUserMedia) {
// 调用用户媒体设备, 访问摄像头
this.getUserMedia(this.videoOption, this.success.bind(this), this.error);
}
else {
alert('不支持访问用户媒体');
}
}
get curVideo() {
return this.video;
}
}
web/examples/tinyYolo/classDemo.es6 0 → 100644
浏览文件 @ 7110db47
import 'babel-polyfill';
import Graph from '../../src/executor/loader';
import IO from '../../src/feed/imageFeed';
import Logger from '../../tools/logger';
window.log = new Logger();
// 统计参数
window.badCases = [];
// import Utils from '../src/utils/utils';
// 获取map表
// import Map from '../test/data/map';
// import demoPic from './bbt1.jpg';
// import demoPic2 from './bbt2.jpg';
// import demoPic3 from './bbt3.jpg';
// import demoPic4 from './bbt4.jpg';
// import demoPic5 from './bbt5.jpg';
// 后处理测试用例
// let tempPic = [demoPic, demoPic2, demoPic3, demoPic4, demoPic5];
/**
* @file model demo 入口文件
* @author wangqun@baidu.com
*
*/
// 模型输出shape
const outputShapes = {
'608': {
from: [19, 19, 25, 1],
to: [19, 19, 5, 5]
},
'320': {
from: [10, 10, 25, 1],
to: [10, 10, 5, 5]
},
'320fused': {
from: [10, 10, 25, 1],
to: [10, 10, 5, 5]
}
};
// 模型feed数据
const feedShape = {
'608': {
fw: 608,
fh: 608
},
'320': {
fw: 320,
fh: 320
},
'320fused': {
fw: 320,
fh: 320
}
};
// 模型路径
const modelPath = {
'608': 'faceModel',
'320': 'facemodel320',
'320fused': 'facemodelfused'
};
const modelType = '320fused';
const path = modelPath[modelType];
// 统计参数
let loaded = false;
let model = {};
window.statistic = [];
const {fw, fh} = feedShape[modelType];
// 第一遍执行比较慢 所以预热一下
async function preheat() {
const io = new IO();
let feed = io.process({
input: video,
params: {
gapFillWith: '#000', // 缩放后用什么填充不足方形部分
targetSize: {
height: fw,
width: fh
},
targetShape: [1, 3, fh, fw], // 目标形状 为了兼容之前的逻辑所以改个名
// shape: [3, 608, 608], // 预设tensor形状
mean: [117.001, 114.697, 97.404], // 预设期望
// std: [0.229, 0.224, 0.225] // 预设方差
}
});
const MODEL_URL = `/${path}/model.json`;
const MODEL_CONFIG = {
dir: `/${path}/`, // 存放模型的文件夹
main: 'model.json', // 主文件
};
loaded = true;
const graphModel = new Graph();
log.start('加载模型');
model = await graphModel.loadGraphModel(MODEL_CONFIG, {
multipart: true,
dataType: 'binary',
binaryOption: {
fileCount: 1, // 切成了多少文件
getFileName(i) { // 获取第i个文件的名称
return 'chunk_0.dat';
}
},
feed
});
log.end('加载模型');
let inst = model.execute({
input: feed
});
};
async function run(input) {
// const input = document.getElementById('mobilenet');
log.start('总耗时');
const io = new IO();
log.start('预处理');
let feed = io.process({
input: input,
params: {
gapFillWith: '#000', // 缩放后用什么填充不足方形部分
targetSize: {
height: fw,
width: fh
},
targetShape: [1, 3, fh, fw], // 目标形状 为了兼容之前的逻辑所以改个名
// shape: [3, 608, 608], // 预设tensor形状
mean: [117.001, 114.697, 97.404], // 预设期望
// std: [0.229, 0.224, 0.225] // 预设方差
}
});
log.end('预处理');
if (!loaded) {
const MODEL_URL = `/${path}/model.json`;
const MODEL_CONFIG = {
dir: `/${path}/`, // 存放模型的文件夹
main: 'model.json', // 主文件
};
loaded = true;
const graphModel = new Graph();
log.start('加载模型');
model = await graphModel.loadGraphModel(MODEL_CONFIG, {
multipart: true,
dataType: 'binary',
binaryOption: {
fileCount: 1, // 切成了多少文件
getFileName(i) { // 获取第i个文件的名称
return 'chunk_0.dat';
}
},
feed
});
log.end('加载模型');
}
log.start('运行耗时');
let inst = model.execute({
input: feed
});
// 其实这里应该有个fetch的执行调用或者fetch的输出
let result = await inst.read();
log.end('后处理-读取数据');
// console.dir(['result', result]);
log.start('后处理-形状调整');
const newData = [];
let newIndex = -1;
const [w, h, c, b] = outputShapes[modelType].from;
// c channel
for (let i = 0; i < c; i++) {
// height channel
for (let j = 0; j < h; j++) {
// width channel
for (let k = 0; k < w; k++) {
// position: (0, 0, 0, 0)
const index = j * (c * h) + k * c + i;
// const index = j * (i * k) + k * i + i;
newData[++newIndex] = result[index];
}
}
}
log.end('后处理-形状调整');
log.start('后处理-画框');
testRun(newData, input);
log.end('后处理-画框');
log.end('后处理');
log.end('总耗时');
};
var image = '';
function selectImage(file) {
if (!file.files || !file.files[0]) {
return;
}
let reader = new FileReader();
reader.onload = function (evt) {
let img = document.getElementById('image');
img.src = evt.target.result;
img.onload = function() {
log.during('每次执行的时间间隔');
run(img);
};
image = evt.target.result;
}
reader.readAsDataURL(file.files[0]);
};
// selectImage
document.getElementById("uploadImg").onchange = function () {
selectImage(this);
};
/* 后处理图片 by zhangmiao06 */
let preTestRun = (index) => {
let img = document.getElementById('image');
img.src = tempPic[index];
img.onload = function() {
testRun(testOutput.data[index], img);
};
};
let testRun = (data, img) => {
// console.log('ori', data);
const {from, to} = outputShapes[modelType];
// let shape = [1, 25, 19, 19];
let shape = [].concat(from).reverse();
// 1.从一维数组到1*25*19*19
let formatData = reshapeMany({
data: data,
reshapeShape: shape
});
// console.log('一维到多维', formatData);
// 2.从1*25*19*19 到 19*19*25*1
let formatData2 = transpose({
data: formatData,
shape: shape,
transposeShape: [2, 3, 1, 0]
});
// console.log('transpose', formatData2);
// 3.从19*19*25*1到19*19*5*5
let formatData3 = reshape({
data: formatData2,
shape: from,
reshapeShape: to
});
// console.log('reshape', formatData3);
// 4.运算
let finalData = handleFinal(formatData3, shape, img);
// console.log('final', finalData);
// 5.处理画布
// handleCanvas(finalData, img);
handleDiv(finalData, img);
};
// sigmoid
let sigmoid = (x) => {
if (x < -100) {
return 0.0;
}
return 1 / (1 + Math.exp(-x));
};
// transpose
let transpose = (data) => {
let shape = data.shape;
let transposeShape = data.transposeShape;
let formatData = data.data;
let formatData2 = [];
for(let n = 0; n < shape[transposeShape[0]]; n++) {
let nData = [];
for(let c = 0; c < shape[transposeShape[1]]; c++) {
let cData = [];
for(let row = 0; row < shape[transposeShape[2]]; row++) {
let rowData = [];
for(let col = 0; col < shape[transposeShape[3]]; col++) {
let tempArr = [n, c, row, col];
let newN = n;
let newC = c;
let newW = row;
let newH = col;
transposeShape.forEach((item, index)=> {
switch(item) {
case 0:
newN = tempArr[index];
break;
case 1:
newC = tempArr[index];
break;
case 2:
newW = tempArr[index];
break;
case 3:
newH = tempArr[index];
}
});
rowData.push(formatData[newN][newC][newW][newH]);
}
cData.push(rowData);
}
nData.push(cData);
}
formatData2.push(nData);
}
return formatData2;
};
// reshape
let reshape = (data) =>{
let formatData2 = data.data;
let shape = data.shape;
let reshapeShape = data.reshapeShape;
// 1.变成一维
let tempData = reshapeOne({
data: formatData2,
shape: shape
});
// 2.变成多维
let formatData3 = reshapeMany({
data: tempData,
reshapeShape: reshapeShape
});
return formatData3;
};
// 变成一维
let reshapeOne = (data) => {
let formatData2 = data.data;
let shape = data.shape;
let tempData = [];
for(let n = 0; n < shape[0]; n++) {
for(let c = 0; c < shape[1]; c++) {
for(let row = 0; row < shape[2]; row++) {
for(let col = 0; col < shape[3]; col++) {
tempData.push(formatData2[n][c][row][col]);
}
}
}
}
return tempData;
};
// 变成多维
let reshapeMany = (data) => {
let tempData = data.data;
let reshapeShape = data.reshapeShape;
let formatData3 = [];
for(let n = 0; n < reshapeShape[0]; n++) {
let nData = [];
for(let c = 0; c < reshapeShape[1]; c++) {
let cData = [];
for(let row = 0; row < reshapeShape[2]; row++) {
let rowData = [];
for(let col = 0; col < reshapeShape[3]; col++) {
let tempN = n * reshapeShape[1] * reshapeShape[2] * reshapeShape[3];
let tempC = c * reshapeShape[2] * reshapeShape[3];
let tempRow = row * reshapeShape[3];
rowData.push(tempData[tempN + tempC + tempRow + col]);
}
cData.push(rowData);
}
nData.push(cData);
}
formatData3.push(nData);
}
return formatData3;
};
let calSize = (img) => {
let w1 = img.width;
let h1 = img.height;
let wh1 = Math.max(w1, h1);
// let factor = 608.0 / wh1;
let factor = fw / wh1;
let width = Math.round(w1 * factor);
let height = Math.round(h1 * factor);
return [w1, h1, width, height];
};
// 处理运算
let handleFinal = (formatData3, shape, img) => {
let finalData = [];
let c = shape[2];
let [w1, h1, width, height] = calSize(img);
let factorX = Math.max(width, height) / width;
let factorY = Math.max(width, height) / height;
let maxProb = 0.0;
let anchors = [[1.603231, 2.094468], [6.041143, 7.080126], [2.882459, 3.518061], [4.266906, 5.178857], [9.041765, 10.66308]];
for(let i = 0; i < shape[2]; i++) {
for(let j = 0; j < shape[3]; j++) {
for(let k = 0; k < anchors.length; k++) {
let [a1, a2, a3, a4, prob] = formatData3[i][j][k];
prob = sigmoid(prob);
if (prob > maxProb && prob >= 0.5) {
let ctx = (j + sigmoid(a1)) / c * factorX;
let cty = (i + sigmoid(a2)) / c * factorY;
let col = Math.exp(a3) * anchors[k][0] / c * factorX;
let row = Math.exp(a4) * anchors[k][1] / c * factorY;
let x = (ctx - (col / 2));
let y = (cty - (row / 2));
finalData.push([x * w1, y * h1, col * w1, row * h1, prob]);
}
}
}
}
return finalData;
};
// 处理画布
let handleCanvas = (finalData, img) => {
let myCanvas = document.getElementById('myCanvas');
let [w1, h1, width, height] = calSize(img);
myCanvas.width = w1;
myCanvas.height = h1;
let ctx = myCanvas.getContext('2d');
ctx.drawImage(img, 0, 0, w1, h1);
finalData.forEach((demoArr,index) => {
let [demoLeft, demoTop, demoWidth, demoHeight, prob] = demoArr;
ctx.beginPath();
ctx.strokeStyle = 'red';
ctx.moveTo(demoLeft, demoTop);
ctx.lineTo(demoLeft + demoWidth, demoTop);
ctx.lineTo(demoLeft + demoWidth, demoTop + demoHeight);
ctx.lineTo(demoLeft, demoTop + demoHeight);
ctx.closePath();
ctx.stroke();
});
};
let handleDiv = (finalData, img) => {
if (finalData.length < 1) {
return false;
}
let myCanvas = document.getElementById('myDiv');
let maxIndex = 0;
if (finalData.length > 1) {
for(let i = 1; i < finalData.length; i++) {
if (finalData[i].prob > finalData[maxIndex].prob) {
maxIndex = i;
}
}
}
let [demoLeft, demoTop, demoWidth, demoHeight, prob] = finalData[maxIndex];
myCanvas.style.width = demoWidth;
myCanvas.style.height = demoHeight;
myCanvas.style.left = demoLeft;
myCanvas.style.top = demoTop;
};
// preTestRun(0);
// run(document.getElementById('pic'));
web/examples/tinyYolo/data.json 0 → 100644
浏览文件 @ 7110db47
此差异已折叠。
web/examples/tinyYolo/index.es6 0 → 100644
浏览文件 @ 7110db47
// import VConsole from 'vconsole';
import 'babel-polyfill';
import Paddle from '../../src/paddle/paddle';
import IO from '../../src/feed/imageFeed';
// import Logger from '../../tools/logger';
// window.log = new Logger();
// // 统计参数
// window.badCases = [];
// 后处理测试用例
// let tempPic = [demoPic, demoPic2, demoPic3, demoPic4, demoPic5];
/**
* @file model demo 入口文件
* @author wangqun@baidu.com
*
*/
// 模型输出shape
const outputShapes = {
'608': {
from: [19, 19, 25, 1],
to: [19, 19, 5, 5]
},
'320': {
from: [10, 10, 25, 1],
to: [10, 10, 5, 5]
},
'320fused': {
from: [10, 10, 25, 1],
to: [10, 10, 5, 5]
},
'tinyYolo': {
from: [10, 10, 25, 1],
to: [10, 10, 5, 5]
}
};
// 模型feed数据
const feedShape = {
'608': {
fw: 608,
fh: 608
},
'320': {
fw: 320,
fh: 320
},
'320fused': {
fw: 320,
fh: 320
},
'tinyYolo': {
fw: 320,
fh: 320
}
};
// 模型路径
const modelPath = {
'tinyYolo': 'model/tinyYolo'
};
const modelType = 'tinyYolo';
const path = modelPath[modelType];
// 统计参数
let loaded = false;
let model = {};
window.statistic = [];
const {fw, fh} = feedShape[modelType];
// 第一遍执行比较慢 所以预热一下
async function run(input) {
// const input = document.getElementById('mobilenet');
//log.start('总耗时');
const io = new IO();
// log.start('预处理');
let feed = io.process({
input: input,
params: {
gapFillWith: '#000', // 缩放后用什么填充不足方形部分
targetSize: {
height: fw,
width: fh
},
targetShape: [1, 3, fh, fw], // 目标形状 为了兼容之前的逻辑所以改个名
// shape: [3, 608, 608], // 预设tensor形状
mean: [117.001, 114.697, 97.404], // 预设期望
// std: [0.229, 0.224, 0.225] // 预设方差
}
});
// log.end('预处理');
if (!loaded) {
const MODEL_CONFIG = {
dir: `/${path}/`, // 存放模型的文件夹
main: 'model.json', // 主文件
};
loaded = true;
const paddle = new Paddle({
urlConf: MODEL_CONFIG,
options: {
multipart: true,
dataType: 'binary',
options: {
fileCount: 1, // 切成了多少文件
getFileName(i) { // 获取第i个文件的名称
return 'chunk_0.dat';
}
}
}
});
model = await paddle.load();
}
let inst = model.execute({
input: feed
});
// 其实这里应该有个fetch的执行调用或者fetch的输出
let result = await inst.read();
// log.end('运行耗时');
// log.end('后处理-读取数据');
console.dir(['result', result]);
//log.start('后处理-形状调整');
const newData = [];
let newIndex = -1;
const [w, h, c, b] = outputShapes[modelType].from;
// c channel
for (let i = 0; i < c; i++) {
// height channel
for (let j = 0; j < h; j++) {
// width channel
for (let k = 0; k < w; k++) {
// position: (0, 0, 0, 0)
const index = j * (c * h) + k * c + i;
// const index = j * (i * k) + k * i + i;
newData[++newIndex] = result[index];
}
}
}
// log.end('后处理-形状调整');
// log.start('后处理-画框');
testRun(newData, input);
// log.end('后处理-画框');
// log.end('后处理');
// log.end('总耗时');
}
var image = '';
function selectImage(file) {
if (!file.files || !file.files[0]) {
return;
}
let reader = new FileReader();
reader.onload = function (evt) {
let img = document.getElementById('image');
img.src = evt.target.result;
img.onload = function() {
//log.during('每次执行的时间间隔');
run(img);
};
image = evt.target.result;
}
reader.readAsDataURL(file.files[0]);
}
// selectImage
document.getElementById("uploadImg").onchange = function () {
selectImage(this);
};
/* 后处理图片 by zhangmiao06 */
let preTestRun = (index) => {
let img = document.getElementById('image');
img.src = tempPic[index];
img.onload = function() {
testRun(testOutput.data[index], img);
};
};
let testRun = (data, img) => {
// console.log('ori', data);
const {from, to} = outputShapes[modelType];
// let shape = [1, 25, 19, 19];
let shape = [].concat(from).reverse();
// 1.从一维数组到1*25*19*19
let formatData = reshapeMany({
data: data,
reshapeShape: shape
});
// console.log('一维到多维', formatData);
// 2.从1*25*19*19 到 19*19*25*1
let formatData2 = transpose({
data: formatData,
shape: shape,
transposeShape: [2, 3, 1, 0]
});
// console.log('transpose', formatData2);
// 3.从19*19*25*1到19*19*5*5
let formatData3 = reshape({
data: formatData2,
shape: from,
reshapeShape: to
});
// console.log('reshape', formatData3);
// 4.运算
let finalData = handleFinal(formatData3, shape, img);
// console.log('final', finalData);
// 5.处理画布
// handleCanvas(finalData, img);
handleDiv(finalData, img);
};
// sigmoid
let sigmoid = (x) => {
if (x < -100) {
return 0.0;
}
return 1 / (1 + Math.exp(-x));
}
// transpose
let transpose = (data) => {
let shape = data.shape;
let transposeShape = data.transposeShape;
let formatData = data.data;
let formatData2 = [];
for(let n = 0; n < shape[transposeShape[0]]; n++) {
let nData = [];
for(let c = 0; c < shape[transposeShape[1]]; c++) {
let cData = [];
for(let row = 0; row < shape[transposeShape[2]]; row++) {
let rowData = [];
for(let col = 0; col < shape[transposeShape[3]]; col++) {
let tempArr = [n, c, row, col];
let newN = n;
let newC = c;
let newW = row;
let newH = col;
transposeShape.forEach((item, index)=> {
switch(item) {
case 0:
newN = tempArr[index];
break;
case 1:
newC = tempArr[index];
break;
case 2:
newW = tempArr[index];
break;
case 3:
newH = tempArr[index];
}
});
rowData.push(formatData[newN][newC][newW][newH]);
}
cData.push(rowData);
}
nData.push(cData);
}
formatData2.push(nData);
}
return formatData2;
};
// reshape
let reshape = (data) =>{
let formatData2 = data.data;
let shape = data.shape;
let reshapeShape = data.reshapeShape;
// 1.变成一维
let tempData = reshapeOne({
data: formatData2,
shape: shape
});
// 2.变成多维
let formatData3 = reshapeMany({
data: tempData,
reshapeShape: reshapeShape
});
return formatData3;
};
// 变成一维
let reshapeOne = (data) => {
let formatData2 = data.data;
let shape = data.shape;
let tempData = [];
for(let n = 0; n < shape[0]; n++) {
for(let c = 0; c < shape[1]; c++) {
for(let row = 0; row < shape[2]; row++) {
for(let col = 0; col < shape[3]; col++) {
tempData.push(formatData2[n][c][row][col]);
}
}
}
}
return tempData;
};
// 变成多维
let reshapeMany = (data) => {
let tempData = data.data;
let reshapeShape = data.reshapeShape;
let formatData3 = [];
for(let n = 0; n < reshapeShape[0]; n++) {
let nData = [];
for(let c = 0; c < reshapeShape[1]; c++) {
let cData = [];
for(let row = 0; row < reshapeShape[2]; row++) {
let rowData = [];
for(let col = 0; col < reshapeShape[3]; col++) {
let tempN = n * reshapeShape[1] * reshapeShape[2] * reshapeShape[3];
let tempC = c * reshapeShape[2] * reshapeShape[3];
let tempRow = row * reshapeShape[3];
rowData.push(tempData[tempN + tempC + tempRow + col]);
}
cData.push(rowData);
}
nData.push(cData);
}
formatData3.push(nData);
}
return formatData3;
};
let calSize = (img) => {
let w1 = img.width;
let h1 = img.height;
let wh1 = Math.max(w1, h1);
// let factor = 608.0 / wh1;
let factor = fw / wh1;
let width = Math.round(w1 * factor);
let height = Math.round(h1 * factor);
return [w1, h1, width, height];
};
// 处理运算
let handleFinal = (formatData3, shape, img) => {
let finalData = [];
let c = shape[2];
let [w1, h1, width, height] = calSize(img);
let factorX = Math.max(width, height) / width;
let factorY = Math.max(width, height) / height;
let maxProb = 0.0;
let anchors = [[1.603231, 2.094468], [6.041143, 7.080126], [2.882459, 3.518061], [4.266906, 5.178857], [9.041765, 10.66308]];
for(let i = 0; i < shape[2]; i++) {
for(let j = 0; j < shape[3]; j++) {
for(let k = 0; k < anchors.length; k++) {
let [a1, a2, a3, a4, prob] = formatData3[i][j][k];
prob = sigmoid(prob);
if (prob > maxProb && prob >= 0.5) {
let ctx = (j + sigmoid(a1)) / c * factorX;
let cty = (i + sigmoid(a2)) / c * factorY;
let col = Math.exp(a3) * anchors[k][0] / c * factorX;
let row = Math.exp(a4) * anchors[k][1] / c * factorY;
let x = (ctx - (col / 2));
let y = (cty - (row / 2));
finalData.push([x * w1, y * h1, col * w1, row * h1, prob]);
}
}
}
}
return finalData;
};
// 处理画布
let handleCanvas = (finalData, img) => {
let myCanvas = document.getElementById('myCanvas');
let [w1, h1, width, height] = calSize(img);
myCanvas.width = w1;
myCanvas.height = h1;
let ctx = myCanvas.getContext("2d");
ctx.drawImage(img, 0, 0, w1, h1);
finalData.forEach((demoArr,index) => {
let [demoLeft, demoTop, demoWidth, demoHeight, prob] = demoArr;
ctx.beginPath();
ctx.strokeStyle="red";
ctx.moveTo(demoLeft, demoTop);
ctx.lineTo(demoLeft + demoWidth, demoTop);
ctx.lineTo(demoLeft + demoWidth, demoTop + demoHeight);
ctx.lineTo(demoLeft, demoTop + demoHeight);
ctx.closePath();
ctx.stroke();
});
};
let handleDiv = (finalData, img) => {
if (finalData.length < 1) {
return false;
}
let myCanvas = document.getElementById('myDiv');
let maxIndex = 0;
if (finalData.length > 1) {
for(let i = 1; i < finalData.length; i++) {
if (finalData[i].prob > finalData[maxIndex].prob) {
maxIndex = i;
}
}
}
let [demoLeft, demoTop, demoWidth, demoHeight, prob] = finalData[maxIndex];
myCanvas.style.width = demoWidth;
myCanvas.style.height = demoHeight;
myCanvas.style.left = demoLeft;
myCanvas.style.top = demoTop;
};
// preTestRun(0);
// run(document.getElementById('pic'));
web/examples/tinyYolo/index.html 0 → 100644
浏览文件 @ 7110db47
<!DOCYTPE html>
<html>
<head>
<meta charset="utf-8">
<title>paddle web demo</title>
<meta name="viewport" content="width=device-width,minimum-scale=1.0,maximum-scale=1.0,user-scalable=no">
<style>
.image-wrap {
position: relative;
}
#myDiv {
position: absolute;
border: 1px solid red;
box-sizing: border-box;
}
</style>
</head>
<body>
<div class="image-wrap">
<img id="mobilenet" />
</div>
<p>原图片</p>
<div class="image-wrap">
<img id="image" src=""/>
<div id="myDiv"></div>
</div>
<p>画布</p>
<canvas id="myCanvas"></canvas>
<br/>
<input type="file" id="uploadImg"/>
<div id="txt"></div>
</body>
<script src="index.es6"></script>
</html>
web/examples/tinyYolo/videoDemo.es6 0 → 100644
浏览文件 @ 7110db47
import 'babel-polyfill';
import Runner from '../src/executor/runner';
import Camera from '../src/executor/camera';
// 调试工具
// import vConsole from 'vconsole';
// const theConsole = new vConsole();
let startBtn = document.getElementById('start');
let stopBtn = document.getElementById('stop')
const runner = new Runner({
// 用哪个模型
modelName: 'separate' // '608' | '320' | '320fused' | 'separate'
});
startBtn.disabled = true;
runner.preheat()
.then(() =>{
startBtn.disabled = false
});
const domElement = document.getElementById('video');
const myCanvas = document.getElementById('myDiv');
const videoSelect = document.getElementById('videoSelect');
let camera = new Camera({
// 用来显示摄像头图像的dom
videoDom: domElement
});
camera.getDevices().then(devices => {
if (devices.length) {
camera.run(devices[0].deviceId);
devices.forEach((element, index) => {
let option = document.createElement('option');
option.value = element.deviceId;
option.text = (index + 1);
videoSelect.appendChild(option);
});
videoSelect.onchange = () => {
camera.run(videoSelect.value);
};
}
else {
camera.run();
}
});
const handleDiv = function (data) {
myCanvas.style.width = (data ? data[0] : 0) + 'px';
myCanvas.style.height = (data ? data[0] : 0) + 'px';
myCanvas.style.left = (data ? data[2] : 0) + 'px';
myCanvas.style.top = (data ? data[3] : 0) + 'px';
}
startBtn.addEventListener('click', function () {
startBtn.disabled = true;
runner.startStream(() => camera.curVideo, handleDiv);
});
stopBtn.addEventListener('click', function () {
startBtn.disabled = false;
runner.stopStream();
});
\ No newline at end of file
web/examples/tinyYolo/videoDemo.html 0 → 100644
浏览文件 @ 7110db47
<!DOCYTPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>识别摄像头里的脸</title>
<style>
body {
margin: 0;
padding: 0;
}
#myDiv {
position: fixed;
border: 1px solid red;
box-sizing: border-box;
}
#video {
background: red;
}
</style>
</head>
<body>
<video id="video">
</video>
<p>
<button id="start">开始识别</button>
<button id="stop">结束</button>
</p>
<select id="videoSelect"></select>
<p id="tips">tips</p>
<div id="myDiv"></div>
<script src="./videoDemo.es6"></script>
</body>
</html>
\ No newline at end of file
web/examples/tinyYolo/x.json 0 → 100644
浏览文件 @ 7110db47
此差异已折叠。
web/examples/yolo/index.es6 0 → 100644
浏览文件 @ 7110db47
/* eslint-disable */
import 'babel-polyfill';
import Paddle from '../../src/paddle/paddle';
import IO from '../../src/feed/imageFeed';
// import Logger from '../../tools/logger';
// window.log = new Logger();
// // 统计参数
// window.badCases = [];
// 后处理测试用例
// let tempPic = [demoPic, demoPic2, demoPic3, demoPic4, demoPic5];
/**
* @file model demo 入口文件
* @author wangqun@baidu.com
*
*/
// 模型输出shape
const outputShapes = {
'608': {
from: [19, 19, 25, 1],
to: [19, 19, 5, 5]
},
'320': {
from: [10, 10, 25, 1],
to: [10, 10, 5, 5]
},
'320fused': {
from: [10, 10, 25, 1],
to: [10, 10, 5, 5]
},
'separate': {
from: [10, 10, 25, 1],
to: [10, 10, 5, 5]
}
};
// 模型feed数据
const feedShape = {
'608': {
fw: 608,
fh: 608
},
'320': {
fw: 320,
fh: 320
},
'320fused': {
fw: 320,
fh: 320
},
'separate': {
fw: 320,
fh: 320
}
};
// 模型路径
const modelPath = {
'separate': 'model/tinyYolo'
};
const modelType = 'separate';
const path = modelPath[modelType];
// 统计参数
let loaded = false;
let model = {};
window.statistic = [];
const {fw, fh} = feedShape[modelType];
// 第一遍执行比较慢 所以预热一下
async function run(input) {
// const input = document.getElementById('mobilenet');
//log.start('总耗时');
const io = new IO();
// log.start('预处理');
let feed = io.process({
input: input,
params: {
gapFillWith: '#000', // 缩放后用什么填充不足方形部分
targetSize: {
height: fw,
width: fh
},
targetShape: [1, 3, fh, fw], // 目标形状 为了兼容之前的逻辑所以改个名
// shape: [3, 608, 608], // 预设tensor形状
mean: [117.001, 114.697, 97.404], // 预设期望
// std: [0.229, 0.224, 0.225] // 预设方差
}
});
// log.end('预处理');
if (!loaded) {
const MODEL_CONFIG = {
dir: `/${path}/`, // 存放模型的文件夹
main: 'model.json', // 主文件
};
loaded = true;
const paddle = new Paddle({
urlConf: MODEL_CONFIG,
options: {
multipart: true,
dataType: 'binary',
options: {
fileCount: 1, // 切成了多少文件
getFileName(i) { // 获取第i个文件的名称
return 'chunk_0.dat';
}
},
feed
}
});
model = await paddle.load();
}
let inst = model.execute({
input: feed
});
// 其实这里应该有个fetch的执行调用或者fetch的输出
let result = await inst.read();
// log.end('运行耗时');
// log.end('后处理-读取数据');
console.dir(['result', result]);
//log.start('后处理-形状调整');
const newData = [];
let newIndex = -1;
const [w, h, c, b] = outputShapes[modelType].from;
// c channel
for (let i = 0; i < c; i++) {
// height channel
for (let j = 0; j < h; j++) {
// width channel
for (let k = 0; k < w; k++) {
// position: (0, 0, 0, 0)
const index = j * (c * h) + k * c + i;
// const index = j * (i * k) + k * i + i;
newData[++newIndex] = result[index];
}
}
}
// log.end('后处理-形状调整');
// log.start('后处理-画框');
testRun(newData, input);
// log.end('后处理-画框');
// log.end('后处理');
// log.end('总耗时');
}
var image = '';
function selectImage(file) {
if (!file.files || !file.files[0]) {
return;
}
let reader = new FileReader();
reader.onload = function (evt) {
let img = document.getElementById('image');
img.src = evt.target.result;
img.onload = function() {
//log.during('每次执行的时间间隔');
run(img);
};
image = evt.target.result;
}
reader.readAsDataURL(file.files[0]);
}
// selectImage
document.getElementById("uploadImg").onchange = function () {
selectImage(this);
};
/* 后处理图片 by zhangmiao06 */
let preTestRun = (index) => {
let img = document.getElementById('image');
img.src = tempPic[index];
img.onload = function() {
testRun(testOutput.data[index], img);
};
};
let testRun = (data, img) => {
// console.log('ori', data);
const {from, to} = outputShapes[modelType];
// let shape = [1, 25, 19, 19];
let shape = [].concat(from).reverse();
// 1.从一维数组到1*25*19*19
let formatData = reshapeMany({
data: data,
reshapeShape: shape
});
// console.log('一维到多维', formatData);
// 2.从1*25*19*19 到 19*19*25*1
let formatData2 = transpose({
data: formatData,
shape: shape,
transposeShape: [2, 3, 1, 0]
});
// console.log('transpose', formatData2);
// 3.从19*19*25*1到19*19*5*5
let formatData3 = reshape({
data: formatData2,
shape: from,
reshapeShape: to
});
// console.log('reshape', formatData3);
// 4.运算
let finalData = handleFinal(formatData3, shape, img);
// console.log('final', finalData);
// 5.处理画布
// handleCanvas(finalData, img);
handleDiv(finalData, img);
};
// sigmoid
let sigmoid = (x) => {
if (x < -100) {
return 0.0;
}
return 1 / (1 + Math.exp(-x));
}
// transpose
let transpose = (data) => {
let shape = data.shape;
let transposeShape = data.transposeShape;
let formatData = data.data;
let formatData2 = [];
for(let n = 0; n < shape[transposeShape[0]]; n++) {
let nData = [];
for(let c = 0; c < shape[transposeShape[1]]; c++) {
let cData = [];
for(let row = 0; row < shape[transposeShape[2]]; row++) {
let rowData = [];
for(let col = 0; col < shape[transposeShape[3]]; col++) {
let tempArr = [n, c, row, col];
let newN = n;
let newC = c;
let newW = row;
let newH = col;
transposeShape.forEach((item, index)=> {
switch(item) {
case 0:
newN = tempArr[index];
break;
case 1:
newC = tempArr[index];
break;
case 2:
newW = tempArr[index];
break;
case 3:
newH = tempArr[index];
}
});
rowData.push(formatData[newN][newC][newW][newH]);
}
cData.push(rowData);
}
nData.push(cData);
}
formatData2.push(nData);
}
return formatData2;
};
// reshape
let reshape = (data) =>{
let formatData2 = data.data;
let shape = data.shape;
let reshapeShape = data.reshapeShape;
// 1.变成一维
let tempData = reshapeOne({
data: formatData2,
shape: shape
});
// 2.变成多维
let formatData3 = reshapeMany({
data: tempData,
reshapeShape: reshapeShape
});
return formatData3;
};
// 变成一维
let reshapeOne = (data) => {
let formatData2 = data.data;
let shape = data.shape;
let tempData = [];
for(let n = 0; n < shape[0]; n++) {
for(let c = 0; c < shape[1]; c++) {
for(let row = 0; row < shape[2]; row++) {
for(let col = 0; col < shape[3]; col++) {
tempData.push(formatData2[n][c][row][col]);
}
}
}
}
return tempData;
};
// 变成多维
let reshapeMany = (data) => {
let tempData = data.data;
let reshapeShape = data.reshapeShape;
let formatData3 = [];
for(let n = 0; n < reshapeShape[0]; n++) {
let nData = [];
for(let c = 0; c < reshapeShape[1]; c++) {
let cData = [];
for(let row = 0; row < reshapeShape[2]; row++) {
let rowData = [];
for(let col = 0; col < reshapeShape[3]; col++) {
let tempN = n * reshapeShape[1] * reshapeShape[2] * reshapeShape[3];
let tempC = c * reshapeShape[2] * reshapeShape[3];
let tempRow = row * reshapeShape[3];
rowData.push(tempData[tempN + tempC + tempRow + col]);
}
cData.push(rowData);
}
nData.push(cData);
}
formatData3.push(nData);
}
return formatData3;
};
let calSize = (img) => {
let w1 = img.width;
let h1 = img.height;
let wh1 = Math.max(w1, h1);
// let factor = 608.0 / wh1;
let factor = fw / wh1;
let width = Math.round(w1 * factor);
let height = Math.round(h1 * factor);
return [w1, h1, width, height];
};
// 处理运算
let handleFinal = (formatData3, shape, img) => {
let finalData = [];
let c = shape[2];
let [w1, h1, width, height] = calSize(img);
let factorX = Math.max(width, height) / width;
let factorY = Math.max(width, height) / height;
let maxProb = 0.0;
let anchors = [[1.603231, 2.094468], [6.041143, 7.080126], [2.882459, 3.518061], [4.266906, 5.178857], [9.041765, 10.66308]];
for(let i = 0; i < shape[2]; i++) {
for(let j = 0; j < shape[3]; j++) {
for(let k = 0; k < anchors.length; k++) {
let [a1, a2, a3, a4, prob] = formatData3[i][j][k];
prob = sigmoid(prob);
if (prob > maxProb && prob >= 0.5) {
let ctx = (j + sigmoid(a1)) / c * factorX;
let cty = (i + sigmoid(a2)) / c * factorY;
let col = Math.exp(a3) * anchors[k][0] / c * factorX;
let row = Math.exp(a4) * anchors[k][1] / c * factorY;
let x = (ctx - (col / 2));
let y = (cty - (row / 2));
finalData.push([x * w1, y * h1, col * w1, row * h1, prob]);
}
}
}
}
return finalData;
};
// 处理画布
let handleCanvas = (finalData, img) => {
let myCanvas = document.getElementById('myCanvas');
let [w1, h1, width, height] = calSize(img);
myCanvas.width = w1;
myCanvas.height = h1;
let ctx = myCanvas.getContext("2d");
ctx.drawImage(img, 0, 0, w1, h1);
finalData.forEach((demoArr,index) => {
let [demoLeft, demoTop, demoWidth, demoHeight, prob] = demoArr;
ctx.beginPath();
ctx.strokeStyle="red";
ctx.moveTo(demoLeft, demoTop);
ctx.lineTo(demoLeft + demoWidth, demoTop);
ctx.lineTo(demoLeft + demoWidth, demoTop + demoHeight);
ctx.lineTo(demoLeft, demoTop + demoHeight);
ctx.closePath();
ctx.stroke();
});
};
let handleDiv = (finalData, img) => {
if (finalData.length < 1) {
return false;
}
let myCanvas = document.getElementById('myDiv');
let maxIndex = 0;
if (finalData.length > 1) {
for(let i = 1; i < finalData.length; i++) {
if (finalData[i].prob > finalData[maxIndex].prob) {
maxIndex = i;
}
}
}
let [demoLeft, demoTop, demoWidth, demoHeight, prob] = finalData[maxIndex];
myCanvas.style.width = demoWidth;
myCanvas.style.height = demoHeight;
myCanvas.style.left = demoLeft;
myCanvas.style.top = demoTop;
};
// preTestRun(0);
// run(document.getElementById('pic'));
/* eslint-enable */
\ No newline at end of file
web/examples/yolo/index.html 0 → 100644
浏览文件 @ 7110db47
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>paddle web demo</title>
<meta name="viewport" content="width=device-width,minimum-scale=1.0,maximum-scale=1.0,user-scalable=no">
<style>
.image-wrap {
position: relative;
}
#myDiv {
position: absolute;
border: 1px solid #f71111;
box-sizing: border-box;
}
</style>
</head>
<body>
<div class="image-wrap">
<img id="mobilenet">
</div>
<p>原图片</p>
<div class="image-wrap">
<img id="image" src="pic.png">
<div id="myDiv"></div>
</div>
<p>画布</p>
<canvas id="myCanvas"></canvas>
<input type="file" id="uploadImg">
<div id="txt"></div>
<script src="index.es6"></script>
</body>
</html>
web/package.json
浏览文件 @ 7110db47
......@@ -4,14 +4,21 @@
"description": "paddle",
"main": "index.js",
"scripts": {
"server": "parcel ./src/index.html",
"testDemo": "parcel ./demo/index.html",
"testSDemo": "parcel ./demo/index.html --port 8125 --https",
"testVideoDemo": "parcel ./demo/videoDemo.html --port 8123 --https",
"mnistdemo": "parcel ./examples/mnist/index.html",
"mobilenet": "parcel ./examples/mobileNet/index.html",
"tinyYolo": "parcel ./examples/tinyYolo/index.html",
"huangfan": "parcel ./examples/huangfan/index.html",
"yolo": "parcel ./examples/yolo/index.html",
"videoDemo": "parcel ./examples/videoDemo.html --port 8123 --https",
"unitTest": "parcel ./test/unitTest.html",
"test": "echo \"Error: no test specified\" && exit 1"
},
"devDependencies": {
"@babel/core": "^7.7.2",
"@babel/preset-env": "^7.7.1",
"axios": "^0.17.1",
"babel-core": "^6.26.3",
"babel-loader": "^8.0.6",
"babel-plugin-transform-class-properties": "^6.24.1",
"babel-plugin-transform-decorators-legacy": "^1.3.5",
"babel-plugin-transform-runtime": "^6.23.0",
......@@ -20,7 +27,8 @@
"babel-preset-react": "^6.24.1",
"babel-preset-stage-0": "^6.24.1",
"babel-runtime": "^6.26.0",
"parcel-bundler": "^1.10.3"
"parcel-bundler": "^1.10.3",
"webpack-cli": "^3.3.6"
},
"keywords": [],
"author": "",
......
web/src/executor/camera.es6
浏览文件 @ 7110db47
......@@ -26,7 +26,7 @@ export default class Camera {
}
// 访问用户媒体设备的兼容方法
run(deviceId) {
run(deviceId, callback) {
if (window.stream) {
window.stream.getTracks().forEach(function (track) {
track.stop();
......@@ -36,7 +36,9 @@ export default class Camera {
let constraints = {
video: {}
};
const success = this.success.bind(this);
const success = stream => {
this.success(stream, callback);
};
const error = this.error.bind(this);
if (this.deviceInfos.length) {
constraints.video.deviceId= {exact: deviceId || this.deviceInfos[0]};
......@@ -63,7 +65,7 @@ export default class Camera {
}
}
success(stream) {
success(stream, callback) {
const domElement = this.video;
// make stream available to console
window.stream = stream;
......@@ -88,6 +90,7 @@ export default class Camera {
domElement.height = $(domElement).height();
}
domElement.play();
callback && callback();
}, false);
}
......
web/src/executor/executor.es6
浏览文件 @ 7110db47
......@@ -82,9 +82,10 @@ export default class GraphExecutor {
// console.log(inputs, outputs);
if (this.type !== 'feed') {
// let time = +Date.now();
log.start(this.opData.iLayer + '-' + this.type);
// log.start(this.opData.iLayer + '-' + this.type);
console.log(this.type, this.opData);
runtime.run(this.type, this.opData, isRendered);
log.end(this.opData.iLayer + '-' + this.type);
// log.end(this.opData.iLayer + '-' + this.type);
// if (runtime.gpu.frameBufferIsComplete().isComplete) {
// var result = runtime.read();
// let res = Array.prototype.slice.call(result);
......
web/src/executor/postProcess.es6
浏览文件 @ 7110db47
......@@ -137,7 +137,7 @@ export default class PostProcess {
this.lastRect = [0, 0, 0, 0]
}
run(data, img, callback) {
run(data, img, callback, canavs) {
let {from, to} = this.modelConfig.outputShapes;
let shape = [].concat(from).reverse();
// 1.从一维数组到1*25*19*19
......@@ -167,7 +167,7 @@ export default class PostProcess {
// console.log('final', finalData);
// 5.处理画布
// finalData.length && handleCanvas(finalData, img);
this.handleDiv(finalData, img, callback);
this.handleDiv(finalData, img, callback, canavs);
}
calSize(img) {
......@@ -213,7 +213,7 @@ export default class PostProcess {
return finalData;
}
handleDiv(finalData, img, callback) {
handleDiv(finalData, img, callback, canavs) {
if (finalData.length < 1) {
callback();
return false;
......@@ -230,7 +230,7 @@ export default class PostProcess {
let [demoLeft, demoTop, demoWidth, demoHeight] = finalData[maxIndex];
if (!isSimilar(this.lastRect, [demoLeft, demoTop, demoWidth, demoHeight])) {
callback([demoWidth, demoHeight,demoLeft, demoTop]);
callback([demoWidth, demoHeight,demoLeft, demoTop], canavs);
};
this.lastRect = [demoLeft, demoTop, demoWidth, demoHeight];
}
......
web/src/executor/runner.es6
浏览文件 @ 7110db47
......@@ -9,7 +9,6 @@
* r.run(document.getElementById('test'));
* });
*/
import IO from '../feed/ImageFeed';
import DataFeed from '../feed/dataFeed';
import Graph from './loader';
......@@ -43,10 +42,12 @@ export default class Runner {
shape: [1, 3, fh, fw]
}];
const MODEL_URL = `/${path}/model.json`;
let dir = `https://mms-graph.cdn.bcebos.com/activity/facegame/paddle/${path}/`;
if (location.href.indexOf('test=1') > -1) {
dir = `/src/view/common/lib/paddle/${path}/`;
}
const MODEL_CONFIG = {
dir: `/${path}/`, // 存放模型的文件夹
// dir: `https://graph.baidu.com/mms/graph/static/asset/dll/${path}/`, // rd测试地址
// dir: `/src/view/common/lib/paddle/dist/${path}/`, // 本地测试地址
dir: dir,
main: 'model.json' // 主文件
};
const graphModel = new Graph();
......@@ -77,8 +78,8 @@ export default class Runner {
console.warn('It\'s better to preheat the model before running.');
await this.preheat();
}
log.start('总耗时'); // eslint-disable-line
log.start('预处理'); // eslint-disable-line
// log.start('总耗时'); // eslint-disable-line
// log.start('预处理'); // eslint-disable-line
let feed;
if (typeof input === 'string') {
const dfIO = new DataFeed();
......@@ -103,13 +104,13 @@ export default class Runner {
}
});
}
log.end('预处理'); // eslint-disable-line
log.start('运行耗时'); // eslint-disable-line
// log.end('预处理'); // eslint-disable-line
// log.start('运行耗时'); // eslint-disable-line
let inst = this.model.execute({
input: feed
});
let result = await inst.read();
log.end('后处理-读取数据'); // eslint-disable-line
// log.end('后处理-读取数据'); // eslint-disable-line
const newData = [];
let newIndex = -1;
const [w, h, c, b] = this.modelConfig.outputShapes.from;
......@@ -126,15 +127,15 @@ export default class Runner {
}
}
}
this.postProcess.run(newData, input, callback);
log.end('后处理'); // eslint-disable-line
this.postProcess.run(newData, input, callback, feed[0].canvas);
// log.end('后处理'); // eslint-disable-line
this.flags.isRunning = false;
log.end('总耗时'); // eslint-disable-line
// log.end('总耗时'); // eslint-disable-line
}
// 传入获取图片的function
async runStream(getMedia, callback) {
await this.run(getMedia(), callback);
await this.run(getMedia, callback);
if (!this.flags.runVideoPaused) {
setTimeout(async () => {
await this.runStream(getMedia, callback);
......
web/src/factory/fshader/factory.es6
浏览文件 @ 7110db47
import ops from './ops';
/**
* @file 工厂类，生成fragment shader
* @author yangmingming
* @author wangqun
*/
export default class Factory {
constructor(opts) {
......
web/src/factory/fshader/ops.es6
浏览文件 @ 7110db47
......@@ -130,6 +130,11 @@ export default {
func: dynamic_func,
confs: dynamic_conf
},
relu6: {
params: dynamic_params,
func: dynamic_func,
confs: dynamic_conf
},
scale: {
params: dynamic_params,
func: dynamic_func,
......
web/src/feed/ImageFeed.es6
浏览文件 @ 7110db47
......@@ -6,6 +6,7 @@
export default class imageFeed {
constructor() {
this.fromPixels2DContext = document.createElement('canvas').getContext('2d');
this.fromPixels2DContext2 = document.createElement('canvas').getContext('2d');
this.defaultWidth = 224;
this.defaultHeight = 224;
this.minPixels = 225;
......@@ -32,7 +33,8 @@ export default class imageFeed {
let output = [];
if (!this.result) {
const [b, c, h, w] = params.targetShape;
this.result = new Float32Array(h * w * 3);
// 计算确定targetShape所需Float32Array占用空间
this.result = new Float32Array(h * w * c);
}
output = this.fromPixels(input, params);
return output;
......@@ -49,14 +51,17 @@ export default class imageFeed {
const vPadding = Math.ceil((sh - height) / 2);
let data = imageData.data;
// channel RGB
let red = [];
let green = [];
let blue = [];
// 平均数
let mean = opt.mean;
// 标准差
let std = opt.std;
// 考虑channel因素获取数据
for (let i = 0; i < data.length; i += 4) {
// img_mean 0.485, 0.456, 0.406
//img_std 0.229, 0.224, 0.225
let index = i / 4;
let vIndex = Math.floor(index / sw);
let hIndex = index - (vIndex * sw) - 1;
......@@ -67,6 +72,7 @@ export default class imageFeed {
blue.push(((data[i + 2] / 255) - mean[2]) / std[2]); // blue
}
}
// 转成 GPU 加速 NCHW 格式
let tmp = green.concat(blue);
return red.concat(tmp);
};
......@@ -78,7 +84,7 @@ export default class imageFeed {
allReshapeToRGB(imageData, opt, scaleSize) {
const {sw, sh} = scaleSize;
const [b, c, h, w] = opt.targetShape;
let data = imageData.data;
let data = imageData.data || imageData;
let mean = opt.mean;
let dataLength = data.length;
// let result = new Float32Array(dataLength * 3);
......@@ -127,6 +133,7 @@ export default class imageFeed {
this.fromPixels2DContext.canvas.height = sh;
this.fromPixels2DContext.drawImage(
image, 0, 0, sw, sh);
this.setInputCanvas(image);
return {sw, sh};
};
......@@ -167,11 +174,26 @@ export default class imageFeed {
image, 0, 0, sw, sh);
// currentPic = this.fromPixels2DContext.canvas.toDataURL();
}
this.setInputCanvas(image);
// window.currentPic = this.fromPixels2DContext.canvas;// test only, demele me
// document.getElementById('p-c').appendChild(this.fromPixels2DContext.canvas);// test only, demele me
return {sw: targetWidth, sh: targetHeight};
}
/**
* 设置原始video画布
* @param image 原始video
*/
setInputCanvas(image) {
// 原始图片宽高
const width = this.pixelWidth;
const height = this.pixelHeight;
// 画布设置
this.fromPixels2DContext2.canvas.width = width;
this.fromPixels2DContext2.canvas.height = height;
this.fromPixels2DContext2.drawImage(image, 0, 0, width, height);
}
/**
* 获取图像内容
* @param pixels
......@@ -179,11 +201,12 @@ export default class imageFeed {
*/
getImageData(pixels, scaleSize) {
const {sw, sh} = scaleSize;
// 复制画布上指定矩形的像素数据
let vals = this.fromPixels2DContext
.getImageData(0, 0, sw, sh);
// crop图像
const width = pixels.width;
const height = pixels.height;
// const width = pixels.width;
// const height = pixels.height;
return vals;
};
......@@ -196,6 +219,7 @@ export default class imageFeed {
let data = imageData.data;
for (let i = 0; i < data.length; i += 4) {
// 3 channel 灰度处理无空间压缩
let avg = (data[i] + data[i + 1] + data[i + 2]) / 3;
data[i] = avg; // red
data[i + 1] = avg; // green
......@@ -206,6 +230,8 @@ export default class imageFeed {
fromPixels(pixels, opt) {
let data;
// 原始video画布数据
let data2;
let scaleSize;
if (pixels instanceof HTMLImageElement || pixels instanceof HTMLVideoElement) {
this.pixelWidth = pixels.naturalWidth || pixels.width;
......@@ -213,10 +239,12 @@ export default class imageFeed {
if (opt.scale) { // 兼容以前的，如果有scale就是短边缩放到scale模式
scaleSize = this.reSize(pixels, opt);
data = this.getImageData(opt, scaleSize);
data2 = this.fromPixels2DContext2.getImageData(0, 0, this.pixelWidth, this.pixelHeight);
}
else if (opt.targetSize) { // 如果有targetSize，就是装在目标宽高里的模式
scaleSize = this.fitToTargetSize(pixels, opt);
data = this.getImageData(opt, scaleSize);
data2 = this.fromPixels2DContext2.getImageData(0, 0, this.pixelWidth, this.pixelHeight);
}
}
......@@ -224,14 +252,15 @@ export default class imageFeed {
data = grayscale(data);
}
if (opt.shape) {
if (opt.reShape) {
data = this.reshape(data, opt, scaleSize);
}
if (opt.targetShape) {
data = this.allReshapeToRGB(data, opt, scaleSize);
}
return [{data: data, shape: opt.shape || opt.targetShape, name: 'image'}];
return [{data: data, shape: opt.shape || opt.targetShape, name: 'image', canvas: data2}];
}
}
/* eslint-enable */
web/src/gpu/gpu.es6
浏览文件 @ 7110db47
......@@ -3,7 +3,7 @@ import VSHADER from '../shader/v_shader';
import VSHADER2 from '../shader/v_shader2';
/**
* @file gpu运算
* @author yangmingming
* @author wangqun@baidu.com, yangmingming@baidu.com
*/
const CONF = {
alpha: false,
......@@ -268,6 +268,7 @@ export default class gpu {
// this.currentTexture = this.textureBuffer[this.textureBufferIndex % 2];
// this.textureBufferIndex = (this.textureBufferIndex + 1) >= 2 ? 0 : 1;
this.currentTexture = this.outTextures[iLayer];
console.log('this.currentTexture', this.currentTexture);
const gl = this.gl;
gl.framebufferTexture2D(gl.FRAMEBUFFER, // The target is always a FRAMEBUFFER.
gl.COLOR_ATTACHMENT0, // We are providing the color buffer.
......@@ -360,9 +361,16 @@ export default class gpu {
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texImage2D(gl.TEXTURE_2D, 0, this.internalFormat, item.width_texture,
item.height_texture, 0,
this.textureFormat, gl.FLOAT, item.data, 0);
gl.texImage2D(gl.TEXTURE_2D,
0,
this.internalFormat,
item.width_texture,
item.height_texture,
0,
this.textureFormat,
gl.FLOAT,
item.data,
0);
}
}
......@@ -404,14 +412,15 @@ export default class gpu {
render(data = [], iLayer = 0, isRendered = false) {
const gl = this.gl;
let that = this;
let textureIndex = 0;
data.forEach(item => {
if (item.type === 'texture') {
this.initTexture(textureIndex, item, iLayer, isRendered);
gl.uniform1i(this.getUniformLoc(item.variable + '_' + item.tensor, iLayer, isRendered), textureIndex++);
that.initTexture(textureIndex, item, iLayer, isRendered);
gl.uniform1i(that.getUniformLoc(item.variable + '_' + item.tensor, iLayer, isRendered), textureIndex++);
}
else if (item.type === 'uniform') {
gl[item.setter](this.getUniformLoc(item.variable + '_' + item.tensor, iLayer, isRendered), item.data);
gl[item.setter](that.getUniformLoc(item.variable + '_' + item.tensor, iLayer, isRendered), item.data);
}
});
// gl.clearColor(.0, .0, .0, 1);
......@@ -437,7 +446,7 @@ export default class gpu {
gl2.bindBuffer(gl2.PIXEL_PACK_BUFFER, buffer);
gl2.getBufferSubData(gl2.PIXEL_PACK_BUFFER, 0, pixels);
gl2.bindBuffer(gl2.PIXEL_PACK_BUFFER, null);
log.start('后处理-readloop');
// log.start('后处理-readloop');
// let result = [];
// let offset = 0;
// for (let h = 0; h < this.height_texture_out; h++) {
......@@ -460,7 +469,7 @@ export default class gpu {
}
// const result = Array.prototype.slice.call(pixels);
// console.dir(['result', result]);
log.end('后处理-readloop');
// log.end('后处理-readloop');
return result;
}
......@@ -517,20 +526,20 @@ export default class gpu {
compute() {
let gl = this.gl;
log.start('后处理-readinside');
// log.start('后处理-readinside');
const tt = +Date.now();
let pixels = new Float32Array(this.width_texture_out * this.height_texture_out * 4);
// gl.pixelStorei(gl.UNPACK_ALIGNMENT, 1);
const tt2 = +Date.now();
gl.readPixels(0, 0, this.width_texture_out, this.height_texture_out, gl.RGBA, gl.FLOAT, pixels, 0);
// console.log('本次读取数据时间是' + (+Date.now() - tt2)+ ',' + (tt2 - tt));
log.end('后处理-readinside');
log.start('后处理-readloop');
// log.end('后处理-readinside');
// log.start('后处理-readloop');
let result = [];
for (let i = 0; i < this.width_texture_out * this.height_texture_out; i++) {
result.push(pixels[4 * i]);
}
log.end('后处理-readloop');
// log.end('后处理-readloop');
return result;
}
......
web/src/executor/loader.es6 → web/src/graph/graph.es6
浏览文件 @ 7110db47
/* eslint-disable */
import GraphExecutor from './executor';
import GraphExecutor from '../executor/executor';
import IO from '../feed/imageFeed';
import Runtime from '../../src/runtime/runtime';
import Runtime from '../runtime/runtime';
import OpData from '../utils/opData';
import Factory from '../factory/fshader/factory';
import Utils from '../utils/utils';
/**
* @file GraphModel，绘制生成model网络
* @file Graph，绘制生成model网络
* @author wangqun@baidu.com
*/
let start = 0;
// 生成factory实例
const factory = new Factory({});
// 获取op的输入配置
const opConfs = factory.getOpConfs();
export default class GraphModel {
constructor(modelGonfig, loadOptions) {
export default class Graph {
constructor(options) {
this.version = '0.0.1';
this.handler = 'io.IOHandler';
this.modelGonfig = modelGonfig;
this.loadOptions = loadOptions;
this.multipart = false;
this.weightMap = '';
this.options = options || {};
// feed数据
this.feed = null;
this.index = 0;
this.feedOp = null;
this.feedItem = null;
this.test = false;
this.isExecuted = false;
// 网络层数
this.iLayer = 0;
// fetch xhr jsonp
this.params = {type: 'fetch'};
// 设置分片加载model
if (this.loadOptions) {
this.multipart = this.loadOptions.multipart;
this.feed = {input: this.loadOptions.feed};
if (loadOptions.dataType === 'binary') {
this.binaryOption = loadOptions.binaryOption;
}
if (this.options && this.options.options && this.options.options.test === true) {
this.test = true;
}
if (!this.loadOptions) {
this.loadOptions = {};
} else {
if (!this.inst) {
// op runner
this.inst = Runtime.init();
factory.setWebglVersion(this.inst.getWebglVersion());
// this.fetchJson(this.modelGonfig.dir + 'x.json').then(data => {
// const [b, c, h, w] = [1, 3, 320, 320];
// const size = data.length;
// const total = 3 * 320 * 320;
// this.testData = new Float32Array(total);
// for (let i = 0; i < size; i++) {
// let j = i / (c * w) | 0;
// let k = i % (c * w);
// let b1 = j / h | 0;
// let h1 = j % h;
// let c1 = k % c;
// let w1 = k / c | 0;
// let l = b1 * (c * h * w) + c1 * (h * w) + h1 * (w) + w1;
// this.testData[i] = data[l];
// }
// });
}
}
fetchOneChunk(path) {
return this.fetch(path).then(request => {
return request.arrayBuffer();
})
}
fetchJson(path) {
return this.fetch(path).then(request => {
return request.json();
})
}
fetchAllData() {
// todo 兼容一下json的模式
let counts = this.binaryOption.fileCount;
let chunkArray = [];
for (let i = 1; i <= counts; i++) {
chunkArray.push(
this.fetchOneChunk(this.modelGonfig.dir + this.binaryOption.getFileName(i))
);
}
console.time('加载时间');
return Promise.all(chunkArray).then(chunks => {
console.timeEnd('加载时间');
let chunksLength = 0;
let f32Array = [];
let float32Chunk;
chunks.forEach(i => {
float32Chunk = new Float32Array(i);
f32Array.push(float32Chunk);
chunksLength += float32Chunk.length;
});
this.allData = new Float32Array(chunksLength);
let offset = 0;
f32Array.forEach(i => {
i.forEach(num => {
this.allData[offset] = num;
offset += 1;
})
});
});
}
traverse (arr) {
const TMP_SCHEME_REGEX = /\.tmp/;
const TMP_REGEX = /\-/;
let marker = 0; // 读到哪个位置了
let len; // 当前op长度
arr.filter(item => {
return item.name
&& item.name.match(TMP_SCHEME_REGEX) === null
&& item.name.match(TMP_REGEX) === null;
})
// .sort((a, b) => {
// if (a.name > b.name) {
// return 1;
// }
// if (a.name < b.name) {
// return -1;
// }
// return 0;
// }) // 按字母顺序排列 在model.json里
.forEach(item => {
len = item.shape.reduce((a, b) => a * b); // 长度为shape的乘积
item.data = this.allData.slice(marker, marker + len);
marker += len;
});
}
fetch(path, params) {
params = params || this.params;
let method = params.method || 'get';
let mode = params.mode || 'cors';
let myHeaders = new Headers();
return fetch(path, {
method: method,
mode: mode,
credentials: 'include',
headers: myHeaders
});
}
fetchModel(params) {
params = params || this.params;
const path = this.modelGonfig.dir + this.modelGonfig.main;
let load = null;
// jsonp请求方式
if (params && params.type === 'jsonp') {
let json;
let s = document.createElement('script');
s.src = path + '&jsonpCallback=fn';
window.fn = function(data) {
json = data;
// console.log(json);
};
//当script被插入文档中时，src中的资源就会开始加载
document.body.appendChild(s);
load = new Promise((resolve, reject) => {
s.onload = function(e) {
resolve(json);
}
s.onerror = function() {
reject(json);
}
});
this.handler = load;
}
// 原生fetch
else if (params.type === 'fetch') {
load = new Promise((resolve, reject) => {
this.fetch(path, params)
.then(response => response.json())
.then(responseData => resolve(responseData))
.then(err => reject(err))
});
this.handler = load;
}
// ajax
else if (params.type === 'xhr') {
this.handler = load;
}
return load;
}
async load() {
let that = this;
const artifacts = this.handler = await this.fetchModel();
if (this.multipart === true) {
await this.fetchAllData()
.then(() => this.traverse(artifacts.vars));
}
const opsMap = this.createOpsMap(artifacts.ops, artifacts.vars);
this.weightMap = this.constructOpsMap(opsMap);
// 生成op数据
this.weightMap.forEach(op => {
const type = op.type;
if (type !== 'feed' && type !== 'fetch') {
that.buildOpData(op);
}
});
return true;
}
buildOpData(op) {
const tensor = this.constructTensor(op);
const opData = new OpData(op.type, tensor.inputs, tensor.outputs, tensor.attrs);
const executor = this.constructExecutor(op);
const opData = new OpData(op.type, executor.inputs, executor.outputs, executor.attrs);
const name = opData.name;
const fsCode = factory.buildShader(name, opData.data);
opData.fsCode = fsCode;
opData.program = this.inst.createProgram(fsCode, opData.tensor['out']);
opData.renderData = opConfs[name].map(elem => {
......@@ -214,6 +57,7 @@ export default class GraphModel {
const tensorData = opData.tensor[item.tensor];
if (item.type === 'texture') {
item.data = tensorData.data;
if (this.feedOp.id === op.id && item.tensor === 'origin') {
item.shape = tensorData.shape;
this.feedItem = item;
......@@ -226,6 +70,7 @@ export default class GraphModel {
}
return item;
});
// console.timeEnd('opData.renderData');
opData.iLayer = this.iLayer++;
op.opData = opData;
......@@ -238,10 +83,11 @@ export default class GraphModel {
return;
}
executor.execute(this.inst, this.isExecuted);
// if (executor.next && start++ < 2) {
if (executor.next) {
const id = executor.next;
const next = this.getTensor(id);
this.execute_(next[0])
this.execute_(next[0]);
}
}
/**
......@@ -262,7 +108,6 @@ export default class GraphModel {
if (this.isExecuted) {
this.updateFeed();
}
let start = +Date.now();
this.execute_(executor[0]);
this.isExecuted = true;
return this.inst;
......@@ -280,13 +125,13 @@ export default class GraphModel {
return this.execute_(inputs, true, this.outputNodes);
}
getTensorAttr(name) {
return this.handler.vars.filter((item, i) => {
return this.data.vars.filter((item, i) => {
if (name === item.name)
return item;
});
}
constructTensor(executor) {
const that = this;
constructExecutor(executor) {
let that = this;
const inputName = executor.inputsName[0];
const input = executor.inputs;
const output = executor.outputs;
......@@ -294,14 +139,22 @@ export default class GraphModel {
output[key] = that.getTensorAttr(output[key][0]);
});
Object.keys(input).forEach(function(key){
if ((key === 'Input') && (inputName === 'pixel')) {
const pixel = that.getTensorAttr(inputName);
const io = new IO();
input[key] = io.fromPixels(data, pixel);
if (that.test && ((key === 'Input') || (key === 'X'))) {
input[key] = that.getTensorAttr(input[key][0]);
that.feedOp = executor;
}
else if ((key === 'Input') && (inputName === 'pixel')) {
// const pixel = that.getTensorAttr(inputName);
// const io = new IO();
// input[key] = io.fromPixels(that.feed, pixel);
input[key] = that.feed.input;
that.feedOp = executor;
}
else if ((key === 'Input') && (inputName === 'image' || inputName === 'x')) {
// that.feed.input[0].data = that.testData;
input[key] = that.feed.input;
that.feedOp = executor;
}
else {
......@@ -309,14 +162,13 @@ export default class GraphModel {
}
});
// console.log(input);
const tensor = {
return {
inputs: input,
outputs: output,
attrs: executor.attrs,
type: executor.type,
next: executor.next
};
return tensor;
}
/**
* Construct Ops Relationship
......@@ -393,26 +245,7 @@ export default class GraphModel {
}
});
}
/**
* Load a graph model given a URL to the model definition.
* @param modelGonfig
* @param options
* @returns {Promise<void>}
*/
async loadGraphModel(modelGonfig, options) {
if (modelGonfig === null) {
// todo saniac 报错提示修改
throw new Error(
'modelGonfig in loadGraphModel() cannot be null. Please provide a url ' +
'or an IOHandler that loads the model');
}
if (options === null) {
options = {};
}
const model = new GraphModel(modelGonfig, options);
await model.load();
return model;
}
/**
* dispose
*/
......
web/src/index.es6 已删除 100644 → 0
浏览文件 @ cbeff2b5
import 'babel-polyfill';
import Graph from './executor/loader';
import IO from './executor/io';
/**
* @file model demo 入口文件
* @author yangmingming@baidu.com
*
*/
// 'http://mms-xr.cdn.bcebos.com/paddle/mnist/model.json'
const MODEL_URL = '../demo/model/model.json';
const graphModel = new Graph();
const model = graphModel.loadGraphModel(MODEL_URL);
const cat = document.getElementById('pic');
const io = new IO();
let inst = model.execute({input: cat});
let res = inst.read();
console.dir(['result', res]);
var fileDownload = require('js-file-download');
fileDownload(res, "result.csv");
web/src/index.html 已删除 100644 → 0
浏览文件 @ cbeff2b5
<!DOCYTPE html>
<html>
<head>
<meta charset="utf-8">
<title>paddle web demo</title>
<meta name="viewport" content="width=device-width,minimum-scale=1.0,maximum-scale=1.0,user-scalable=no">
</head>
<body>
<div><img id="pic" src="data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/wAALCAAcABwBAREA/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/9oACAEBAAA/APn+vTPDHwP8TeJ9DtdXiuLCzt7kbo0uWcOU7NgKRgjkc81i+O/hvrPgW8xco1zp7ELHfIm1HYqCRjJIPUc9cHFcbSgEnABJ9BXaafH8Rrrw3NpdjBrkmjohLQLE/l7c5OOPUHgV6Fcw3um/sxXNt4hZo7qW5X7FDdLtlRfOU7QG5zgSH/dPpXhFel/Bzxj4a8H6vfzeILZy86ILe6WLzPI27i3HUZ+XkA9PQ16Pc/Hfw7pM91LaXusa20wDRxSQRQww9eAdob35DfWuNg+Ny67Dfab430SDUNLuQxjW2UK8BwcAZPPOPmyCOvPSvH6KKKK//9k=" ></div>
</body>
<script src="index.es6"></script>
</html>
web/src/loader/loader.es6 0 → 100644
浏览文件 @ 7110db47
/* eslint-disable */
/**
* @file loader，model加载器
* @author wangqun@baidu.com
*/
export default class Loader {
constructor(modelGonfig, options) {
this.version = '0.0.1';
this.data = {};
this.modelGonfig = modelGonfig;
this.options = options;
this.multipart = false;
this.test = false;
// fetch xhr jsonp
this.params = {type: 'fetch'};
// 设置分片加载model
if (this.options) {
this.multipart = this.options.multipart;
if (options.dataType === 'binary') {
this.binaryOption = options.options;
this.dataType = options.dataType;
}
if (options.test) {
this.test = true;
}
}
if (!this.loadOptions) {
this.loadOptions = {};
}
}
fetchOneChunk(path) {
return this.fetch(path).then(request => {
return request.arrayBuffer();
})
}
fetchJson(path) {
return this.fetch(path).then(request => {
return request.json();
})
}
fetchChunks() {
let counts = this.binaryOption.fileCount;
let chunkArray = [];
for (let i = 1; i <= counts; i++) {
chunkArray.push(
this.fetchOneChunk(this.modelGonfig.dir + this.binaryOption.getFileName(i))
);
}
// console.time('加载时间');
return Promise.all(chunkArray).then(chunks => {
// console.timeEnd('加载时间');
let chunksLength = 0;
let f32Array = [];
let float32Chunk;
chunks.forEach(i => {
float32Chunk = new Float32Array(i);
f32Array.push(float32Chunk);
chunksLength += float32Chunk.length;
});
this.allData = new Float32Array(chunksLength);
let offset = 0;
f32Array.forEach(i => {
i.forEach(num => {
this.allData[offset] = num;
offset += 1;
})
});
});
}
fetchData(name) {
const path = this.modelGonfig.dir + name + '.json';
let load = new Promise((resolve, reject) => {
fetch(path, {
method: 'get', mode: 'cors', credentials: "include",
headers: { 'Content-Type': 'application/json;charset=utf-8'}})
.then(response => response.json())
.then(responseData => resolve(responseData))
.then(err => reject(err))
})
return load;
}
async fetchAllDate (arr) {
const TMP_SCHEME_REGEX = /\.tmp/;
const TMP_REGEX = /\-/;
let requesterArr = arr.map(item => {
if (item.name
&& item.name.match(TMP_SCHEME_REGEX) === null
&& item.name.match(TMP_REGEX) === null) {
return this.fetchData(item.name).then(data => item.data = data);
}
return Promise.resolve();
});
return Promise.all(requesterArr);
}
traverse (arr) {
const TMP_SCHEME_REGEX = /\.tmp/;
const TMP_REGEX = /\-/;
let marker = 0; // 读到哪个位置了
let len; // 当前op长度
arr.filter(item => {
return item.name
&& item.name.match(TMP_SCHEME_REGEX) === null
&& item.name.match(TMP_REGEX) === null;
})
.forEach(item => {
len = item.shape.reduce((a, b) => a * b); // 长度为shape的乘积
item.data = this.allData.slice(marker, marker + len);
marker += len;
});
}
fetch(path, params) {
params = params || this.params;
let method = params.method || 'get';
let mode = params.mode || 'no-cors';
let myHeaders = new Headers();
return fetch(path, {
method: method,
// mode: mode,
// credentials: 'include',
headers: myHeaders
});
}
fetchModel(params) {
params = params || this.params;
const path = this.modelGonfig.dir + this.modelGonfig.main;
let load = null;
// jsonp请求方式
if (params && params.type === 'jsonp') {
let json;
let s = document.createElement('script');
s.src = path + '&jsonpCallback=fn';
window.fn = function(data) {
json = data;
// console.log(json);
};
//当script被插入文档中时，src中的资源就会开始加载
document.body.appendChild(s);
load = new Promise((resolve, reject) => {
s.onload = function(e) {
resolve(json);
}
s.onerror = function() {
reject(json);
}
});
this.data = load;
}
// 原生fetch
else if (params.type === 'fetch') {
load = new Promise((resolve, reject) => {
this.fetch(path, params)
.then(response => response.json())
.then(responseData => resolve(responseData))
.then(err => reject(err))
});
this.data = load;
}
// ajax
else if (params.type === 'xhr') {
this.data = load;
}
return load;
}
async load() {
let that = this;
const artifacts = this.data = await this.fetchModel();
if (this.multipart === true) {
if (this.dataType === 'binary') {
await this.fetchChunks()
.then(() => this.traverse(artifacts.vars));
}
else {
await that.fetchAllDate(artifacts.vars);
}
}
return artifacts;
}
}
/* eslint-enable */
web/src/paddle/paddle.es6 0 → 100644
浏览文件 @ 7110db47
/* eslint-disable */
import 'babel-polyfill';
import Loader from '../loader/loader';
import Graph from '../graph/graph';
/**
* @file paddle对象，负责加载模型和执行在线推理
* @author wangqun@baidu.com
*/
export default class Paddle {
constructor(options) {
this.version = '0.0.1';
this.loader = '';
this.options = options;
this.graph = '';
this.multipart = false;
// feed数据
this.feed = null;
this.index = 0;
this.feedOp = null;
this.feedItem = null;
this.test = false;
this.isExecuted = false;
// 网络层数
this.iLayer = 0;
// fetch xhr jsonp
this.params = {type: 'fetch'};
}
async load() {
if (this.options === null) {
// todo saniac 报错提示修改
throw new Error(
'modelGonfig in loadGraphModel() cannot be null. Please provide a url ' +
'or an IOHandler that loads the model');
}
const model = new Loader(this.options.urlConf, this.options.options);
await model.load();
this.preGraph(model);
return this;
}
preGraph (artifacts) {
let that = this;
const graph = new Graph(that.options);
that.graph = graph;
that.graph.data = artifacts.data;
const opsMap = that.graph.createOpsMap(that.graph.data.ops, that.graph.data.vars);
that.graph.weightMap = that.graph.constructOpsMap(opsMap);
}
/**
* Executes inference for the model for given input tensors.
* @param inputs
* @param outputs
* @returns {*}
*/
execute(inputs) {
debugger;
let that = this;
this.feed = this.graph.feed = inputs;
// 生成op数据
if (!this.graph.isExecuted) {
this.graph.weightMap.forEach(op => {
const type = op.type;
if (type !== 'feed' && type !== 'fetch') {
console.log(op.type);
that.graph.buildOpData(op);
}
});
}
this.graph.execute(inputs);
return this.graph.inst;
}
updateFeed() {
this.graph.feedItem.data = this.graph.feed.input[0].data;
// Utils.img2texture(this.graph.feedItem);
}
/**
* dispose
*/
dispose() {
this.graph.dispose();
}
}
/* eslint-enable */
web/src/runtime/runtime.es6
浏览文件 @ 7110db47
......@@ -3,7 +3,7 @@ import Gpu from '../gpu/gpu';
import getMaxUniforms from '../test/getMaxUniforms';
/**
* @file gpu运行时
* @author yangmingming
* @author wangqun@baidu.com, yangmingming@baidu.com
*
*/
export default {
......@@ -73,10 +73,10 @@ export default {
async read() {
const pbo = this.gpu.createPBO();
await this.gpu.createAndWaitForFence();
log.end('运行耗时');
log.start('后处理');
// log.end('运行耗时');
// log.start('后处理');
// 其实这里应该有个fetch的执行调用或者fetch的输出
log.start('后处理-读取数据');
// log.start('后处理-读取数据');
// 开始读数据
return this.gpu.downloadFoat32TensorFromBuffer(pbo);
},
......
web/src/shader/atom/common_func.es6
浏览文件 @ 7110db47
......@@ -10,9 +10,14 @@ float prelu(float x, float p, float b) {
if (x < 0.0) {
result = x * p;
}
return result;
}
float relu6(float x, float threshold, float b) {
float result = max(0.0,x);
result = min(result,threshold);
return result;
}
float leakyRelu(float x, float p, float b) {
float result = max(x, x * p);
return result;
......@@ -32,4 +37,6 @@ float softmax(float x, float p, float b) {
float result = exp(x) / (10.0 * exp(x));
return result;
}
`;
web/src/shader/atom/getArrayIndexFromTensorPos.es6
浏览文件 @ 7110db47
......@@ -2,6 +2,7 @@
/**
* @file 公共方法
* @author yangmingming
*
*/
export default `
......
web/src/shader/atom/getOutputTensorPos.es6
浏览文件 @ 7110db47
......@@ -2,11 +2,13 @@
/**
* @file 公共方法
* @author yangmingming
* @desc 获取输出tensor的坐标
*/
export default `
ivec4 getOutputTensorPos() {
// 获取原始长度
vec2 outCoord = moveTexture2PosToReal_texture_out(vCoord.xy);
// 材质体系转tensor体系坐标位置
int x = int(outCoord.x / float(channel_out));
int c = int(mod(outCoord.x, float(channel_out)));
int y = int(mod(outCoord.y, float(height_shape_out)));
......
web/src/shader/atom/getPixelsFromTexturePos.es6
浏览文件 @ 7110db47
......@@ -2,6 +2,7 @@
/**
* @file 公共方法
* @author yangmingming
* desc 根据当前材质坐标位置获取值
*/
// 获取材质中的像素
export default `
......
web/src/shader/atom/getValueFromTensorPos.es6
浏览文件 @ 7110db47
......@@ -2,8 +2,10 @@
/**
* @file 公共方法
* @author yangmingming
* desc 根据tensor坐标获取这个tensor位置的值
*/
export default `
// 根据tensor坐标获取这个tensor位置的值
float getValueFromTensorPos_TENSOR_NAME(int r, int g, int b, int a) {
vec4 pixels = TEXTURE2D(texture_TENSOR_NAME,
vec2(
......@@ -11,9 +13,10 @@ float getValueFromTensorPos_TENSOR_NAME(int r, int g, int b, int a) {
(float(r * height_shape_TENSOR_NAME + b) + 0.5) / float(height_texture_TENSOR_NAME)
)
);
// 只用了r通道
return pixels.r;
}
// 紧凑型布局根据tensor坐标获取这个tensor位置的值
float getValueFromTensorPosLimit_TENSOR_NAME(int r, int g, int b, int a) {
float halfW = ceil(float(width_shape_TENSOR_NAME) / 2.0);
int x = int(mod(float(a), halfW));
......
web/src/shader/atom/getValueFromTensorPosPacked.es6
浏览文件 @ 7110db47
......@@ -2,6 +2,7 @@
/**
* @file 公共方法
* @author yangmingming
* desc packed布局 根据tensor坐标获取这个tensor位置的值
*/
export default `
float getValueFromTensorPosPacked_TENSOR_NAME(int r, int g, int b, int a) {
......
web/src/shader/atom/getValueFromTexturePos.es6
浏览文件 @ 7110db47
......@@ -2,6 +2,7 @@
/**
* @file 公共方法
* @author yangmingming
* desc 根据材质坐标获取这个材质位置的值
*/
// TEXTURE_NAME, tensor name
// 获取材质中的数据
......
web/src/shader/atom/moveTexture2PosToReal.es6
浏览文件 @ 7110db47
......@@ -2,18 +2,24 @@
/**
* @file 公共方法
* @author yangmingming
* desc 坐标转化
*/
// TEXTURE_NAME, 材质name
// 材质坐标转化成真实尺寸坐标
export default `
vec2 _2d_shape_TEXTURE_NAME = vec2(float(width_TEXTURE_NAME), float(height_TEXTURE_NAME));
// vec2 moveTexture2PosToReal_TEXTURE_NAME(vec2 v) {
// return v * _2d_shape_TEXTURE_NAME;
// // vec2 v2;
// // v2.x = v.x * float(width_TEXTURE_NAME);
// // v2.y = v.y * float(height_TEXTURE_NAME);
// // return v2;
// }
vec2 moveTexture2PosToReal_TEXTURE_NAME(vec2 v) {
return v * _2d_shape_TEXTURE_NAME;
// vec2 v2;
// v2.x = v.x * float(width_TEXTURE_NAME);
// v2.y = v.y * float(height_TEXTURE_NAME);
// return v2;
vec2 v2;
v2.x = v.x * float(width_TEXTURE_NAME);
v2.y = v.y * float(height_TEXTURE_NAME);
return v2;
}
`;
web/src/shader/atom/prefix.es6
浏览文件 @ 7110db47
......@@ -11,7 +11,7 @@ export default `
precision mediump float;
precision mediump int;
#endif
varying vec2 vCoord;
void setOutput(float result) {
gl_FragColor.r = result;
}
......
web/src/shader/atom/scale.es6
浏览文件 @ 7110db47
/* eslint-disable */
/**
* @file 激活函数
* @author yangmingming
* @author wangqun@baidu.com
*/
export default `
float scale(float x, float p, float b) {
......
web/src/shader/atom/softmax.es6
浏览文件 @ 7110db47
/* eslint-disable */
/**
* @file softmax激活函数
* @author yangmingming
* @author wangqun
*/
export default `
float softmax(float x, float p, float b) {
......
web/src/shader/batchnorm/conf.es6
浏览文件 @ 7110db47
......@@ -26,14 +26,12 @@ export default {
'HEIGHT_TEXTURE_ORIGIN',
'CHANNEL_ORIGIN',
'TOTAL_SHAPE_ORIGIN',
'WIDTH_SHAPE_OUT',
'HEIGHT_SHAPE_OUT',
'WIDTH_TEXTURE_OUT',
'HEIGHT_TEXTURE_OUT',
'CHANNEL_OUT',
'OFFSET_Y_OUT',
'EPSILON',
'WIDTH_TEXTURE_SCALE',
'HEIGHT_TEXTURE_SCALE',
......@@ -55,4 +53,4 @@ export default {
type: 'texture'
}
]
};
};
\ No newline at end of file
web/src/shader/batchnorm/main.es6
浏览文件 @ 7110db47
/* eslint-disable */
/**
* @file softmax主函数
* @author yangmingming
* @file batchnorm主函数
* @author wangqun
*/
export default `
// start函数
void main(void) {
// 输出数据
ivec4 oPos = getOutputTensorPos();
float o = getValueFromTensorPos_origin(oPos);
float o = getValueFromTensorPos_origin(oPos.r, oPos.g, oPos.b, oPos.a);
// 归一化数据
vec4 scale = getPixelsFromTexturePos_texture_scale(vec2((float(int(oPos.g)) + 0.5) / float(width_texture_scale), 0.0));
float x = (o - scale[3]) / sqrt(scale[2] + epsilon);
float res = scale[0] * x + scale[1];
setOutput(res);
}
`;
`;
\ No newline at end of file
web/src/shader/batchnorm/params.es6
浏览文件 @ 7110db47
......@@ -12,13 +12,11 @@ const int width_texture_origin = WIDTH_TEXTURE_ORIGIN;
const int height_texture_origin = HEIGHT_TEXTURE_ORIGIN;
const int channel_origin = CHANNEL_ORIGIN;
const int total_shape_origin = TOTAL_SHAPE_ORIGIN;
// 计算数据
const float epsilon = float(EPSILON);
const int width_texture_scale = WIDTH_TEXTURE_SCALE;
const int height_texture_scale = HEIGHT_TEXTURE_SCALE;
// 输入数据
uniform sampler2D texture_origin;
uniform sampler2D texture_scale;
`;
`;
\ No newline at end of file
web/src/shader/mul/conf.es6
浏览文件 @ 7110db47
/* eslint-disable */
graph.es6/* eslint-disable */
/**
* @file mul的配置文件
* @author yangmingming zhangmiao06
......
web/src/shader/softmax/main.es6
浏览文件 @ 7110db47
......@@ -6,12 +6,14 @@
export default `
// start函数
void main(void) {
float res = 0.0;
vec4 v4 = getPixelsFromTexturePos_texture_origin(vCoord);
vec2 onePixel = vec2(1.0 / float(width_texture_origin), 1.0 / float(height_texture_origin));
float total = 0.0;
float maxValue = getPixelsFromTexturePos_texture_origin(onePixel).r;
int number = 0;
vec4 pixels;
vec4 result;
// 求最大
for (int i = 0; i < height_texture_origin; i++) {
for (int j = 0; j < width_texture_origin; j++) {
......@@ -50,6 +52,9 @@ void main(void) {
}
}
}
gl_FragColor = exp(v4 - vec4(maxValue, maxValue, maxValue, maxValue)) / vec4(total, total, total, total) ;
outColor = exp(v4 - vec4(maxValue, maxValue, maxValue, maxValue)) / vec4(total, total, total, total);
// res = result.a;
// setOutput(res);
}
`;
web/src/shader/v_shader.es6
浏览文件 @ 7110db47
/* eslint-disable */
/**
* @file 顶点文件
* @author yangmingming
* @author wangqun
* @desc  顶点坐标系转换，适配webgl1
*/
export default `
attribute vec4 position;
varying vec2 vCoord;
void main() {
vCoord.x = (position.x + 1.0) / 2.0;
vCoord.y = (position.y + 1.0) / 2.0;
......
web/src/shader/v_shader2.es6
浏览文件 @ 7110db47
/* eslint-disable */
/**
* @file 顶点文件,webgl 2.0
* @author yangmingming
* @author wangqun
* @desc  顶点坐标系转换，适配webgl2
*/
export default `#version 300 es
in vec4 position;
out vec2 vCoord;
void main() {
vCoord.x = (position.x + 1.0) / 2.0;
vCoord.y = (position.y + 1.0) / 2.0;
......
web/src/utils/opData.es6
浏览文件 @ 7110db47
......@@ -3,7 +3,7 @@ import Utils from './utils';
import Tensor from './tensor';
/**
* @file op的数据对象
* @author yangmingming
* @author wangqun, yangmingming
*
*/
const keys = [
......@@ -38,7 +38,7 @@ const shaderAttrs = {
'pooling_type': 'type_pool'
}
};
// model的名字和paddle web的tensor名字mapping
// model的名字和paddleJS的tensor名字mapping
const tensorName = {
'input': 'origin',
'x': 'origin',
......@@ -80,6 +80,10 @@ const opBehavior = {
'transToPrelu',
'needBatch'
],
relu6: [
'transToRelu6',
'needBatch'
],
leaky_relu: [
'transToLeakyrelu',
'needBatch'
......@@ -87,17 +91,25 @@ const opBehavior = {
mul: [
'reshape',
'needBatch'
],
softmax: [
]
};
const mergeType = 'conv2d-elementwise_add';
export default class OpData {
constructor(name, input = {}, output = {}, attrs = {}) {
console.log('now in constructor');
console.dir(name);
console.dir(input);
console.dir(output);
this.realName = name;
this.name = name;
this.attrs = attrs;
// 检查是否是融合op
this.checkIsMerge();
// 是否忽略当前当前op, 使用dropout
// dropout是指在深度学习网络的训练过程中,对于神经网络单元,按照一定的概率将其暂时从网络中丢弃。
this.isPass = this.checkIsPass();
if (this.isPass) {
this.input = input;
......@@ -172,6 +184,9 @@ export default class OpData {
}
});
// console.dir(['tensors', this.tensor]);
// console.log('now in buildTensor show this and tensorData');
// console.log(this);
// console.log(tensorData);
}
buildAttrs() {
......@@ -283,10 +298,14 @@ export default class OpData {
item.notTensor = true;
}
});
return;
// mobilenet model
// todo: 默认y的shape length是1, 以后需要实现通用版本
console.log('2. x and y is ');
console.log(x);
console.log(y);
let shape = Utils.getBroadcastShapeInPaddle(x.shape, y.shape, this.attrs['axis']);
// 填充shape数据
if (small.shape.length === 1) {
......@@ -316,6 +335,11 @@ export default class OpData {
this.data['active_function'] = 'prelu';
}
transToRelu6(tensorData = []) {
this.data['multi_value'] = this.attrs['threshold'];
this.data['active_function'] = 'relu6';
}
transToLeakyrelu(tensorData = []) {
this.data['multi_value'] = this.attrs.alpha;
this.data['active_function'] = 'leakyRelu';
......@@ -347,6 +371,7 @@ export default class OpData {
mergeTensor(tensorData = []) {
// 融合scale、bias、variance、mean
let constants = ['scale', 'bias', 'variance', 'mean'];
let result = {};
let data = [];
......@@ -354,13 +379,18 @@ export default class OpData {
result[tensor.tensorName] = tensor;
result[tensor.tensorName + 'Index'] = index;
});
for (let i = 0; i < result[constants[0]].shape[0]; i++) {
data.push(result[constants[0]].data[i]);
data.push(result[constants[1]].data[i]);
data.push(result[constants[2]].data[i]);
data.push(result[constants[3]].data[i]);
}
for (let i = 0; i < result[constants[0]].shape[0]; i++) {
data.push(result[constants[0]].data[i]);
data.push(result[constants[1]].data[i]);
data.push(result[constants[2]].data[i]);
data.push(result[constants[3]].data[i]);
}
tensorData[result[constants[0] + 'Index']].data = data;
for (let i = 0; i < constants.length; i++){
tensorData[result[constants[i] + 'Index']].data = result[constants[i]].data;
}
// 充分利用shader空间
tensorData[result[constants[0] + 'Index']].notCompressed = true;
tensorData[result[constants[0] + 'Index']].shape[0] *= 4;
......
web/src/utils/tensor.es6
浏览文件 @ 7110db47
/* eslint-disable */
import Utils from './utils';
/**
* @file Tensor类
* @author yangmingming
* @author wangqun, yangmingming
*/
export default class Tensor {
constructor(opts = {}) {
......@@ -33,6 +32,7 @@ export default class Tensor {
// tensor数据
let data;
if (opts.type === 'image' || opts.type === 'x') {
console.log('image', this.data);
this.data = opts.data;
}
else if (opts.data && opts.data.length) {
......@@ -42,18 +42,27 @@ export default class Tensor {
let c = shape[1];
let h = shape[2];
let w = shape[3];
for (let i = 0; i < opts.data.length; i++) {
let j = i / (c * w) | 0;
let k = i % (c * w);
let b1 = j / h | 0;
let h1 = j % h;
let c1 = k % c;
let w1 = k / c | 0;
let l = b1 * (c * h * w) + c1 * (h * w) + h1 * (w) + w1;
data[i] = opts.data[l];
if (w) {
for (let i = 0; i < opts.data.length; i++) {
let j = i / (c * w) | 0;
let k = i % (c * w);
let b1 = j / h | 0;
let h1 = j % h;
let c1 = k % c;
let w1 = k / c | 0;
let l = b1 * (c * h * w) + c1 * (h * w) + h1 * (w) + w1;
data[i] = opts.data[l];
}
this.data = data;
}
else {
if (opts.data.length > this.total) {
opts.data = opts.data.slice(0, this.total);
}
this.data = new Float32Array(opts.data);
debugger;
}
this.data = data;
} else {
// batchnorm的scale
this.shape_texture = [4, 1, this.total / 4];
......@@ -158,4 +167,3 @@ export default class Tensor {
}
}
}
/* eslint-enable */
web/src/utils/utils.es6
浏览文件 @ 7110db47
/**
* @file 工具类
* @author yangmingming
* @author wangqun, yangmingming
*/
/* eslint-disable */
export default {
// todo: 适用2维矩阵乘法，以后实现通用版本
getReshapeInPaddle(inputShape = [], counterShape = [], outShape = []) {
......@@ -109,10 +108,10 @@ export default {
* @return {{shape: *[], zeroNumber: number}} {Object} texture信息
*/
getTextureInfoFromTensorShape(shape = [], isPacked = false) {
let b = shape[0];
let c = shape[1];
let h = shape[2];
let w = shape[3];
let b = shape[0] || 1;
let c = shape[1] || 1;
let h = shape[2] || 1;
let w = shape[3] || 1;
let height = b * h;
let width = c * w;
let offsetX = 0;
......@@ -179,12 +178,7 @@ export default {
let l = b1 * (c * h * w) + c1 * (h * w) + h1 * (w) + w1;
data[offset] = renderData.data[l];
offset += 4;
// data.push(renderData.data[l]);
// data.push(0);
// data.push(0);
// data.push(0);
}
renderData.data = data;
}
};
/* eslint-enable */
web/test/README_cn.md 0 → 100644
浏览文件 @ 7110db47
# PaddleJS Examples
百度PaddleJS使用现成的 JavaScript 模型或转换 Paddle 模型以在浏览器中运行。
## 演示
目前Web项目运行TinyYolo模型可以达到30ms以内，对于一般的实时场景已经足够应对。
### 模块化
## 浏览器覆盖面
* PC: Chrome
* Mac: Chrome
* Android: Baidu App and QQ Browser
## 构建部署
```bash
cd web # 进入根目录
npm i # 安装依赖
mkdir dist # 创建资源目录
cd dist # 进入资源目录
git clone https://github.com/DerekYangMing/Paddle-Web-Models.git # 获取模型
mv Paddle-Web-Models/separablemodel . # 移动模型到制定地点
cd .. # 返回根目录
npm run tinyYolo # 启动 tinyYolo 在线推理服务
```
## 如何预览 demo
1. 在浏览器中打开url: https://localhost:8123/
2. 点击【开始检测】按钮。
3. 将人脸对准摄像头，没有问题的话，可以正常检测到人脸。
## 效果
![image](./tinyYolo/demoshow.png)
web/test/data/map.json 0 → 100644
浏览文件 @ 7110db47
此差异已折叠。
web/test/data/model.test.batchnorm.json 0 → 100644
浏览文件 @ 7110db47
此差异已折叠。
web/test/data/model.test.conv2d.json 0 → 100644
浏览文件 @ 7110db47
{
"ops": [
{
"attrs": {
"Scale_in": 1.0,
"Scale_in_eltwise": 1.0,
"Scale_out": 1.0,
"Scale_weights": [
1.0
],
"data_format": "AnyLayout",
"dilations": [
1,
1
],
"exhaustive_search": false,
"force_fp32_output": false,
"fuse_relu": false,
"fuse_relu_before_depthwise_conv": false,
"fuse_residual_connection": false,
"groups": 2,
"is_test": 1,
"paddings": [
0,
0
],
"strides": [
1,
1
],
"use_cudnn": true,
"use_mkldnn": false,
"workspace_size_MB": 4096
},
"inputs": {
"Filter": [
"conv2d_0.w_0"
],
"Input": [
"pixel"
]
},
"outputs": {
"Output": [
"conv2d_0.tmp_0"
]
},
"type": "conv2d"
}
],
"vars": [
{
"data": [
0, 1, 0, 1, 0, 1, 0, 1, 0,
0, 1, 0, 1, 0, 1, 0, 1, 0,
0, 1, 0, 1, 0, 1, 0, 1, 0,
0, 1, 0, 1, 0, 1, 0, 1, 0,
0, 2, 0, 2, 0, 2, 0, 2, 0,
0, 2, 0, 2, 0, 2, 0, 2, 0,
0, 2, 0, 2, 0, 2, 0, 2, 0,
0, 2, 0, 2, 0, 2, 0, 2, 0
],
"name": "pixel",
"persistable": 0,
"shape": [
1,
8,
3,
3
]
},
{
"data": [
0, 1, 0, 1,
0, 1, 0, 1,
0, 1, 0, 1,
0, 1, 0, 1,
0, 2, 0, 2,
0, 2, 0, 2,
0, 2, 0, 2,
0, 2, 0, 2,
0, 2, 0, 2,
0, 2, 0, 2,
0, 2, 0, 2,
0, 2, 0, 2,
0, 2, 0, 2,
0, 2, 0, 2,
0, 2, 0, 2,
0, 2, 0, 2
],
"name": "conv2d_0.w_0",
"persistable": 1,
"shape": [
4,
4,
2,
2
]
},
{
"data": [
],
"name": "conv2d_1.b_0",
"persistable": 1,
"shape": [
50
]
},
{
"data": [
4, 4, 4, 4,
8, 8, 8, 8,
16, 16, 16, 16,
16, 16, 16, 16
],
"name": "conv2d_0.tmp_0",
"persistable": 0,
"shape": [
1,
4,
2,
2
]
},
]
}
web/test/data/model.test.depthwise_conv2d.json 0 → 100644
浏览文件 @ 7110db47
此差异已折叠。
web/test/data/model.test.elementwise_add.json 0 → 100644
浏览文件 @ 7110db47
此差异已折叠。
web/test/data/model.test.json 0 → 100644
浏览文件 @ 7110db47
此差异已折叠。
web/test/data/model.test.mul.json 0 → 100644
浏览文件 @ 7110db47
此差异已折叠。
web/test/data/model.test.pool2d.json 0 → 100644
浏览文件 @ 7110db47
此差异已折叠。
web/test/data/model.test.relu.json 0 → 100644
浏览文件 @ 7110db47
此差异已折叠。
web/test/data/model.test.scale.json 0 → 100644
浏览文件 @ 7110db47
{
"ops": [
{
"attrs": {
"bias": 0.0,
"bias_after_scale": true,
"scale": 1.0
},
"inputs": {
"X": [
"fc_0.tmp_2"
]
},
"outputs": {
"Out": [
"scale_0.tmp_0"
]
},
"type": "scale"
}
],
"vars": [
{
"data": [
1.6038109389511792e-28,
1.6038109389511792e-28,
1.6038109389511792e-28,
1.6038109389511792e-28,
1.6038109389511792e-28,
1.6038109389511792e-28,
1.6038109389511792e-28,
1.6038109389511792e-28,
1.0,
1.6504194778766687e-16
],
"name": "fc_0.tmp_2",
"persistable": 0,
"shape": [
10
]
},
{
"data": [
1.6038109389511792e-28,
1.6038109389511792e-28,
1.6038109389511792e-28,
1.6038109389511792e-28,
1.6038109389511792e-28,
1.6038109389511792e-28,
1.6038109389511792e-28,
1.6038109389511792e-28,
1.0,
1.6504194778766687e-16
],
"name": "scale_0.tmp_0",
"persistable": 0,
"shape": [
10
]
}
]
}
web/test/data/model.test.softmax.json 0 → 100644
浏览文件 @ 7110db47
{
"ops": [
{
"attrs": {
"data_format": "AnyLayout",
"is_test": 1,
"use_cudnn": false,
"use_mkldnn": false
},
"inputs": {
"X": [
"softmax_0.tmp_0"
]
},
"outputs": {
"Out": [
"softmax_0.tmp_2"
]
},
"type": "softmax"
}
],
"vars": [
{
"data":
[0.0320586 , 0.08714432, 0.23688282, 0.64391426]
,
"name": "softmax_0.tmp_2",
"persistable": 0,
"shape": [2, 3, 4]
},
{
"data": [2.0, 3.0, 4.0, 5.0]
,
"name": "softmax_0.tmp_0",
"persistable": 0,
"shape": [2, 3, 4]
}
]
}
web/test/testUtils/diff.js 0 → 100644
浏览文件 @ 7110db47
此差异已折叠。
web/test/testUtils/testUtils.es6 0 → 100644
浏览文件 @ 7110db47
import 'babel-polyfill';
import Paddle from '../../src/paddle/paddle';
const unitPath = {
'conv2d': 'model.test.conv2d.json',
'batchnorm': 'model.test.batchnorm.json',
'mul': 'model.test.mul.json',
'pool2d': 'model.test.pool2d.json',
'relu': 'model.test.relu.json',
'scale': 'model.test.scale.json',
'softmax': 'model.test.softmax.json',
'relu6' : 'model.test.relu6.json'
};
// 制定运行的 op
const modelType = 'softmax';
const unitData = unitPath[modelType];
let Diff = require('./diff');
let datas;
let otherResult;
let output
async function run() {
const path = 'test/unitData';
const MODEL_CONFIG = {
dir: `/${path}/`, // 存放模型的文件夹
main: unitData, // 主文件
};
const paddle = new Paddle({
urlConf: MODEL_CONFIG,
options: {
test: true
}
});
let model = await paddle.load();
datas = model.graph.data;
output = deepCopy(datas);
// 测试单元
model.graph.weightMap.forEach(op => {
const type = op.type;
if (type !== 'feed' && type !== 'fetch') {
console.log(op.type);
model.graph.buildOpData(op);
}
});
const executor = model.graph.weightMap;
let inst = model.graph.execute_(executor[0]);
let result = model.graph.inst.read();
console.dir(['result', result]);
var one = model.graph.inst.read();
// var other = getResult('conv2d');
console.log('one');
console.log(one);
console.log('other');
}
run();
function deepCopy (data) {
return JSON.parse(JSON.stringify(data));
}
// let output = deepCopy(datas);
let getTensor = function(id, times = 1) {
let find = 0;
let data = datas.ops.filter((item, idx) => {
if (id === item.type) {
++find;
if (find === times) {
return true;
}
}
});
return getInputs(data[0]);
};
let getInputs = function(data) {
Object.keys(data.inputs).forEach(function(key){
data.inputs[key] = getValue(data.inputs[key][0], datas);
});
Object.keys(data.outputs).forEach(function(key){
let out = getValue(data.outputs[key][0], datas)
data.outputs[key] = out;
otherResult = out[0].data;
});
return data;
};
let getResult = function(id) {
let data = output.ops.filter((item, idx) => {
if (id === item.type) {
return true;
}
});
return getoutputs(data[0]);
};
let getoutputs = function(data) {
let otherResult;
Object.keys(data.outputs).forEach(function(key){
let out = getValue(data.outputs[key][0], output);
otherResult = out[0].data;
});
return otherResult;
};
let getValue = function(name, datas) {
return datas.vars.filter((item, idx) => {
if (name === item.name) {
return item;
}
});
};
// // 测试单元
// let item = getTensor('conv2d');
let func = function (model) {
// console.log(other);
// var one = inst.read();
// var other = getResult('softmax');
// var color ='';
// var span = null;
// var diff = Diff.diffChars(one.toString(), other.toString()),
// display = document.getElementById('display'),
// fragment = document.createDocumentFragment();
//
// diff.forEach(function(part){
// // green for additions, red for deletions
// // grey for common parts
// color = part.added ? 'green' :
// part.removed ? 'red' : 'grey';
// span = document.createElement('span');
// span.style.color = color;
// span.appendChild(document
// .createTextNode(part.value));
// fragment.appendChild(span);
// });
//
// display.appendChild(fragment);
};
web/test/unitTest.es6 0 → 100644
浏览文件 @ 7110db47
import 'babel-polyfill';
import units from './units/units';
let qs = require('qs');
/**
* @file 入口文件
* @author wangqun@baidu.com
*
*/
// 引入 op
const FSHADER_CON2D = require('../src/shader/f_elementwise_conv2d3_shader.c');
const shapeA = [1, 3, 256, 256];
const shapeB = [3];
const imgUrl = require('./data/banana.jpeg');
let shapeAData;
let shapeBData;
let inst;
const matrix = units.mockOrigin();
const filter = units.mockFilter();
// 原始张量，上下左右1个单位的padding，步长是1
let conf = {
'filter_size_width': 3,
'filter_size_height': 3,
'origin_size_width': matrix.sx,
'origin_size_height': matrix.sx,
'out_size_width': 3,
'out_size_height': 3,
'stride_horizontal': 1,
'stride_vertical': 1,
'pad_left': 1,
'pad_top': 1,
'dilation_horizontal': 2,
'dilation_vertical': 2
}
units.init(conf, FSHADER_CON2D).then(instance => {
if (!instance || typeof instance === 'string') {
throw new Error(instance || '不支持float texture');
}
inst = instance;
}).then(() => {
console.dir(['卷积核', filter]);
console.dir(['origin data', matrix.data]);
// 执行conv2d
inst.compute(filter, matrix.data, 'conv2d');
}).then(() => {
// 读取结果
const result = inst.read();
console.dir(['conv2d的执行结果', result]);
let input = {
filter: filter,
origin: matrix.data,
};
Object.assign(input, conf);
console.dir(['完整input', input]);
// console.dir(['完整输入和输出', params]);
inst.getResult('pool2d', input, result);
}).catch(err => {
console.log('-----------error---------' + err);
});
web/test/unitTest.html 0 → 100644
浏览文件 @ 7110db47
此差异已折叠。
web/test/units/units.es6 0 → 100644
浏览文件 @ 7110db47
此差异已折叠。
web/tools/logger.es6 已删除 100644 → 0
浏览文件 @ cbeff2b5
此差异已折叠。
web/tools/toBinaryFile.py 已删除 100644 → 0
浏览文件 @ cbeff2b5
此差异已折叠。
web/webpack.config.js 已删除 100644 → 0
浏览文件 @ cbeff2b5
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或