未验证 提交 5c67846c 编写于 作者: M manasars 提交者: GitHub

CVAT-3D Milestone2 (#2645)

* CVAT-3D Updated the Mime Types with Bin Support, added dependency of open3D

* CVAT-3D Added additional column as Dimension for engine_task table and created a relatedfiles table for PCD to Image mapping.

* Added Support for 3D file Upload in BIN and PCD.

* Added Dimension attribute defaulting to 2D for importer and exporter.

* Added props passing for dimension attribute, filtering of import, Migration Scripts and Dimension attribute for MpegChunk Writers

* Modified code as per review comments

* Updated Unit test cases for 3D task creation

* Refactored Dimension Enum in UI and backend code

* Resolving conflicts

* Updated Unit Test Case

* Refactored TaskDimension to DimensionType, Simplified usage of Dimension accross classes

* Removing manually created test files

* Removing old pcd mime-type mapping

* Added test files generated by synthetic data using open3d

* Merged with develop branch latest changes

* Added libraries required for open3d

* Added files

* Added synthethic pcd,bin and img test files

* Modified test file name

* Trigger travis ci

* Modified test case to ignore 3D preview images

* Trigger notification

* Deleting DS Store files

* Modified test cases as per review comments

* Checking pre-commit hook

* Fixed Lint issues - precommit hook verification

* Added changes for CVAT-3D Milestone2 changes - Frame Navigation, photo context hide and show

* Modified changes

* Added canvas3D for 3D Perspective

* Added missing files

* Added code to get image context for 3D view

* Codacy check for stylesheet

* Modified frame navigantion for 3D View

* Modified style for context-image

* Trigger notification

* Added Support for 3D file Upload in BIN and PCD.

* Added props passing for dimension attribute, filtering of import, Migration Scripts and Dimension attribute for MpegChunk Writers

* Modified code as per review comments

* Refactored Dimension Enum in UI and backend code

* Merged with develop branch latest changes

* Added files

* Added changes for CVAT-3D Milestone2 changes - Frame Navigation, photo context hide and show

* Modified changes

* Added canvas3D for 3D Perspective

* Added missing files

* Added code to get image context for 3D view

* Codacy check for stylesheet

* Modified frame navigantion for 3D View

* Modified style for context-image

* Changed cvat-data lint issues

* Modified to use opencv as per review comments

* Removed unwanted imports

* Fixed css and added usage of hooks

* Merged Develop branch code

* Removed unused data structures

* Removed unused data structures

* Refactored unused data structures

* Added three js dependency in cvat ui package-lock.json

* Merged develop branch code and refactored code

* Fixed snyk issue

* Modified Camera Icon in photo-context

* Update icons.tsx

* Remove unused svg file

* Modified changelog file

Co-authored-by: cdp <cdp123>
上级 71e2ddbb
......@@ -10,6 +10,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- CVAT-3D: support lidar data on the server side (<https://github.com/openvinotoolkit/cvat/pull/2534>)
- CVAT-3D: Load all frames corresponding to the job instance
(<https://github.com/openvinotoolkit/cvat/pull/2645>)
- Intelligent scissors with OpenCV javascript (<https://github.com/openvinotoolkit/cvat/pull/2689>)
### Changed
......
......@@ -20,6 +20,7 @@ RUN apk add python3 g++ make
# Install dependencies
COPY cvat-core/package*.json /tmp/cvat-core/
COPY cvat-canvas/package*.json /tmp/cvat-canvas/
COPY cvat-canvas3d/package*.json /tmp/cvat-canvas3d/
COPY cvat-ui/package*.json /tmp/cvat-ui/
COPY cvat-data/package*.json /tmp/cvat-data/
......@@ -35,6 +36,10 @@ RUN npm ci
WORKDIR /tmp/cvat-canvas/
RUN npm ci
# Install cvat-canvas dependencies
WORKDIR /tmp/cvat-canvas3d/
RUN npm ci
# Install cvat-ui dependencies
WORKDIR /tmp/cvat-ui/
RUN npm ci
......@@ -42,6 +47,7 @@ RUN npm ci
# Build source code
COPY cvat-data/ /tmp/cvat-data/
COPY cvat-core/ /tmp/cvat-core/
COPY cvat-canvas3d/ /tmp/cvat-canvas3d/
COPY cvat-canvas/ /tmp/cvat-canvas/
COPY cvat-ui/ /tmp/cvat-ui/
RUN npm run build
......
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
module.exports = {
env: {
node: true,
},
parserOptions: {
parser: '@typescript-eslint/parser',
ecmaVersion: 6,
},
plugins: ['@typescript-eslint', 'import'],
extends: [
'plugin:@typescript-eslint/recommended',
'airbnb-typescript/base',
'plugin:import/errors',
'plugin:import/warnings',
'plugin:import/typescript',
],
rules: {
'@typescript-eslint/no-explicit-any': 0,
'@typescript-eslint/indent': ['warn', 4],
'no-plusplus': 0,
'no-restricted-syntax': [
0,
{
selector: 'ForOfStatement',
},
],
'max-len': ['error', { code: 120 }],
'no-continue': 0,
'func-names': 0,
'no-console': 0, // this rule deprecates console.log, console.warn etc. because 'it is not good in production code'
'lines-between-class-members': 0,
'import/prefer-default-export': 0, // works incorrect with interfaces
'newline-per-chained-call': 0, // makes code uglier
},
settings: {
'import/resolver': {
node: {
extensions: ['.ts', '.js', '.json'],
},
},
},
};
# Module CVAT-CANVAS-3D
## Description
The CVAT module written in TypeScript language.
It presents a canvas to viewing, drawing and editing of 3D annotations.
## Versioning
If you make changes in this package, please do following:
- After not important changes (typos, backward compatible bug fixes, refactoring) do: `npm version patch`
- After changing API (backward compatible new features) do: `npm version minor`
- After changing API (changes that break backward compatibility) do: `npm version major`
## Commands
- Building of the module from sources in the `dist` directory:
```bash
npm run build
npm run build -- --mode=development # without a minification
```
### API Methods
```ts
interface Canvas3d {
html(): HTMLDivElement;
setup(frameData: any): void;
fitCanvas(): void;
mode(): Mode;
isAbleToChangeFrame(): boolean;
render(): void;
}
```
### WEB
```js
// Create an instance of a canvas
const canvas = new window.canvas.Canvas3d();
console.log('Version ', window.canvas.CanvasVersion);
console.log('Current mode is ', window.canvas.mode());
// Put canvas to a html container
htmlContainer.appendChild(canvas.html());
canvas.fitCanvas();
```
此差异已折叠。
{
"name": "cvat-canvas3d",
"version": "0.0.1",
"description": "Part of Computer Vision Annotation Tool which presents its canvas3D library",
"main": "src/canvas3d.ts",
"scripts": {
"build": "tsc && webpack --config ./webpack.config.js",
"server": "nodemon --watch config --exec 'webpack-dev-server --config ./webpack.config.js --mode=development --open'"
},
"author": "Intel",
"license": "MIT",
"devDependencies": {
"@babel/cli": "^7.5.5",
"@babel/core": "^7.5.5",
"@babel/plugin-proposal-class-properties": "^7.8.3",
"@babel/plugin-proposal-optional-chaining": "^7.11.0",
"@babel/preset-env": "^7.5.5",
"@babel/preset-typescript": "^7.3.3",
"@types/node": "^12.6.8",
"@typescript-eslint/eslint-plugin": "^1.13.0",
"@typescript-eslint/parser": "^1.13.0",
"babel-loader": "^8.0.6",
"css-loader": "^3.4.2",
"dts-bundle-webpack": "^1.0.2",
"eslint": "^6.1.0",
"eslint-config-airbnb-typescript": "^4.0.1",
"eslint-config-typescript-recommended": "^1.4.17",
"eslint-plugin-import": "^2.18.2",
"node-sass": "^4.14.1",
"nodemon": "^1.19.4",
"postcss-loader": "^3.0.0",
"postcss-preset-env": "^6.7.0",
"sass-loader": "^8.0.2",
"style-loader": "^1.0.0",
"typescript": "^3.5.3",
"webpack": "^4.44.2",
"webpack-cli": "^3.3.6",
"webpack-dev-server": "^3.11.0"
},
"dependencies": {
"three": "^0.125.0"
}
}
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
module.exports = {
parser: false,
plugins: {
'postcss-preset-env': {
browsers: '> 2.5%', // https://github.com/browserslist/browserslist
},
},
};
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
import pjson from '../../package.json';
import { Canvas3dController, Canvas3dControllerImpl } from './canvas3dController';
import { Canvas3dModel, Canvas3dModelImpl, Mode } from './canvas3dModel';
import { Canvas3dView, Canvas3dViewImpl } from './canvas3dView';
import { Master } from './master';
const Canvas3dVersion = pjson.version;
interface Canvas3d {
html(): any;
setup(frameData: any): void;
isAbleToChangeFrame(): boolean;
fitCanvas(): void;
mode(): Mode;
render(): void;
}
class Canvas3dImpl implements Canvas3d {
private model: Canvas3dModel & Master;
private controller: Canvas3dController;
private view: Canvas3dView;
public constructor() {
this.model = new Canvas3dModelImpl();
this.controller = new Canvas3dControllerImpl(this.model);
this.view = new Canvas3dViewImpl(this.model, this.controller);
}
public html(): any {
return this.view.html();
}
public render(): void {
this.view.render();
}
public setup(frameData: any): void {
this.model.setup(frameData);
}
public mode(): Mode {
return this.model.mode;
}
public isAbleToChangeFrame(): boolean {
return this.model.isAbleToChangeFrame();
}
public fitCanvas(): void {
this.model.fitCanvas(this.view.html().clientWidth, this.view.html().clientHeight);
}
}
export { Canvas3dImpl as Canvas3d, Canvas3dVersion };
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
import { Canvas3dModel, Mode } from './canvas3dModel';
export interface Canvas3dController {
mode: Mode;
}
export class Canvas3dControllerImpl implements Canvas3dController {
private model: Canvas3dModel;
public constructor(model: Canvas3dModel) {
this.model = model;
}
public set mode(value: Mode) {
this.model.mode = value;
}
public get mode(): Mode {
return this.model.mode;
}
}
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
import { MasterImpl } from './master';
export interface Size {
width: number;
height: number;
}
export interface Image {
renderWidth: number;
renderHeight: number;
imageData: ImageData | CanvasImageSource;
}
export interface DrawData {
enabled: boolean;
initialState?: any;
redraw?: number;
}
export enum FrameZoom {
MIN = 0.1,
MAX = 10,
}
export enum UpdateReasons {
IMAGE_CHANGED = 'image_changed',
OBJECTS_UPDATED = 'objects_updated',
FITTED_CANVAS = 'fitted_canvas',
DRAW = 'draw',
SELECT = 'select',
CANCEL = 'cancel',
DATA_FAILED = 'data_failed',
}
export enum Mode {
IDLE = 'idle',
DRAG = 'drag',
RESIZE = 'resize',
DRAW = 'draw',
EDIT = 'edit',
INTERACT = 'interact',
}
export interface Canvas3dModel {
mode: Mode;
setup(frameData: any): void;
isAbleToChangeFrame(): boolean;
fitCanvas(width: number, height: number): void;
}
export class Canvas3dModelImpl extends MasterImpl implements Canvas3dModel {
private data: {
canvasSize: Size;
image: Image | null;
imageID: number | null;
imageOffset: number;
imageSize: Size;
drawData: DrawData;
mode: Mode;
exception: Error | null;
};
public constructor() {
super();
this.data = {
canvasSize: {
height: 0,
width: 0,
},
image: null,
imageID: null,
imageOffset: 0,
imageSize: {
height: 0,
width: 0,
},
drawData: {
enabled: false,
initialState: null,
},
mode: Mode.IDLE,
exception: null,
};
}
public setup(frameData: any): void {
if (this.data.imageID !== frameData.number) {
if ([Mode.EDIT, Mode.DRAG, Mode.RESIZE].includes(this.data.mode)) {
throw Error(`Canvas is busy. Action: ${this.data.mode}`);
}
}
this.data.imageID = frameData.number;
frameData
.data((): void => {
this.data.image = null;
this.notify(UpdateReasons.IMAGE_CHANGED);
})
.then((data: Image): void => {
if (frameData.number !== this.data.imageID) {
// already another image
return;
}
this.data.imageSize = {
height: frameData.height as number,
width: frameData.width as number,
};
this.data.image = data;
this.notify(UpdateReasons.IMAGE_CHANGED);
})
.catch((exception: any): void => {
this.data.exception = exception;
this.notify(UpdateReasons.DATA_FAILED);
throw exception;
});
}
public set mode(value: Mode) {
this.data.mode = value;
}
public get mode(): Mode {
return this.data.mode;
}
public isAbleToChangeFrame(): boolean {
const isUnable = [Mode.DRAG, Mode.EDIT, Mode.RESIZE, Mode.INTERACT].includes(this.data.mode)
|| (this.data.mode === Mode.DRAW && typeof this.data.drawData.redraw === 'number');
return !isUnable;
}
public fitCanvas(width: number, height: number): void {
this.data.canvasSize.height = height;
this.data.canvasSize.width = width;
this.data.imageOffset = Math.floor(
Math.max(this.data.canvasSize.height / FrameZoom.MIN, this.data.canvasSize.width / FrameZoom.MIN),
);
this.notify(UpdateReasons.FITTED_CANVAS);
this.notify(UpdateReasons.OBJECTS_UPDATED);
}
}
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
import * as THREE from 'three';
import { PCDLoader } from 'three/examples/jsm/loaders/PCDLoader';
import { Canvas3dController } from './canvas3dController';
import { Listener, Master } from './master';
import { Canvas3dModel, UpdateReasons, Mode } from './canvas3dModel';
export interface Canvas3dView {
html(): HTMLDivElement;
render(): void;
}
export class Canvas3dViewImpl implements Canvas3dView, Listener {
private controller: Canvas3dController;
private renderer: any;
private scene: any;
private camera: any;
private set mode(value: Mode) {
this.controller.mode = value;
}
private get mode(): Mode {
return this.controller.mode;
}
public constructor(model: Canvas3dModel & Master, controller: Canvas3dController) {
this.controller = controller;
this.mode = Mode.IDLE;
this.scene = new THREE.Scene();
this.scene.background = new THREE.Color(0x000000);
// setting up the camera and adding it in the scene
this.camera = new THREE.PerspectiveCamera(50, window.innerWidth / window.innerHeight, 1, 500);
this.camera.position.set(-15, 0, 4);
this.camera.up.set(0, 0, 1);
this.camera.lookAt(0, 0, 0);
this.scene.add(this.camera);
this.renderer = new THREE.WebGLRenderer({ antialias: true });
this.renderer.setPixelRatio(window.devicePixelRatio);
this.renderer.setSize(window.innerWidth, window.innerHeight);
model.subscribe(this);
}
public notify(model: Canvas3dModel & Master, reason: UpdateReasons): void {
if (reason === UpdateReasons.IMAGE_CHANGED) {
const loader = new PCDLoader();
this.clearScene();
const objectURL = URL.createObjectURL(model.data.image.imageData);
loader.load(objectURL, this.addScene.bind(this));
URL.revokeObjectURL(objectURL);
const event: CustomEvent = new CustomEvent('canvas.setup');
this.renderer.domElement.dispatchEvent(event);
}
}
private clearScene(): void {
for (let i = this.scene.children.length - 1; i >= 0; i--) {
this.scene.remove(this.scene.children[i]);
}
}
private addScene(points: any): void {
// eslint-disable-next-line no-param-reassign
points.material.size = 0.03;
// eslint-disable-next-line no-param-reassign
points.material.color = new THREE.Color(0x0000ff);
this.scene.add(points);
}
public render(): void {
this.renderer.render(this.scene, this.camera);
}
public html(): any {
return this.renderer.domElement;
}
}
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
const BASE_GRID_WIDTH = 2;
export default {
BASE_GRID_WIDTH,
};
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
export interface Master {
subscribe(listener: Listener): void;
unsubscribe(listener: Listener): void;
unsubscribeAll(): void;
notify(reason: string): void;
}
export interface Listener {
notify(master: Master, reason: string): void;
}
export class MasterImpl implements Master {
private listeners: Listener[];
public constructor() {
this.listeners = [];
}
public subscribe(listener: Listener): void {
this.listeners.push(listener);
}
public unsubscribe(listener: Listener): void {
for (let i = 0; i < this.listeners.length; i++) {
if (this.listeners[i] === listener) {
this.listeners.splice(i, 1);
}
}
}
public unsubscribeAll(): void {
this.listeners = [];
}
public notify(reason: string): void {
for (const listener of this.listeners) {
listener.notify(this, reason);
}
}
}
{
"compilerOptions": {
"baseUrl": ".",
"emitDeclarationOnly": true,
"module": "es6",
"target": "es6",
"noImplicitAny": true,
"preserveConstEnums": true,
"declaration": true,
"resolveJsonModule": true,
"esModuleInterop": true,
"moduleResolution": "node",
"declarationDir": "dist/declaration",
"paths": {
"cvat-canvas.node": ["dist/cvat-canvas3d.node"]
}
},
"include": ["src/typescript/*.ts"]
}
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
// eslint-disable-next-line @typescript-eslint/no-var-requires
const path = require('path');
// eslint-disable-next-line @typescript-eslint/no-var-requires
const DtsBundleWebpack = require('dts-bundle-webpack');
const nodeConfig = {
target: 'node',
mode: 'production',
devtool: 'source-map',
entry: './src/typescript/canvas3d.ts',
output: {
path: path.resolve(__dirname, 'dist'),
filename: 'cvat-canvas3d.node.js',
library: 'canvas3d',
libraryTarget: 'commonjs',
},
resolve: {
extensions: ['.ts', '.js', '.json'],
},
module: {
rules: [
{
test: /\.ts$/,
exclude: /node_modules/,
use: {
loader: 'babel-loader',
options: {
plugins: [
'@babel/plugin-proposal-class-properties',
'@babel/plugin-proposal-optional-chaining',
],
presets: [['@babel/preset-env'], ['@babel/typescript']],
sourceType: 'unambiguous',
},
},
},
{
test: /\.(css|scss)$/,
exclude: /node_modules/,
use: [
'style-loader',
{
loader: 'css-loader',
options: {
importLoaders: 2,
},
},
'postcss-loader',
'sass-loader',
],
},
],
},
plugins: [
new DtsBundleWebpack({
name: 'cvat-canvas3d.node',
main: 'dist/declaration/src/typescript/canvas3d.d.ts',
out: '../cvat-canvas3d.node.d.ts',
}),
],
};
const webConfig = {
target: 'web',
mode: 'production',
devtool: 'source-map',
entry: {
'cvat-canvas3d': './src/typescript/canvas3d.ts',
},
output: {
path: path.resolve(__dirname, 'dist'),
filename: '[name].[contenthash].js',
library: 'canvas3d',
libraryTarget: 'window',
},
devServer: {
contentBase: path.join(__dirname, 'dist'),
compress: false,
inline: true,
port: 3000,
},
resolve: {
extensions: ['.ts', '.js', '.json'],
},
module: {
rules: [
{
test: /\.ts$/,
exclude: /node_modules/,
use: {
loader: 'babel-loader',
options: {
plugins: ['@babel/plugin-proposal-class-properties'],
presets: [
[
'@babel/preset-env',
{
targets: '> 2.5%', // https://github.com/browserslist/browserslist
},
],
['@babel/typescript'],
],
sourceType: 'unambiguous',
},
},
},
{
test: /\.scss$/,
exclude: /node_modules/,
use: [
'style-loader',
{
loader: 'css-loader',
options: {
importLoaders: 2,
},
},
'postcss-loader',
'sass-loader',
],
},
],
},
plugins: [
new DtsBundleWebpack({
name: 'cvat-canvas3d',
main: 'dist/declaration/src/typescript/canvas3d.d.ts',
out: '../cvat-canvas3d.d.ts',
}),
],
};
module.exports = [webConfig, nodeConfig];
// Copyright (C) 2019-2020 Intel Corporation
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
......@@ -341,6 +341,7 @@
constructor(size, chunkSize, stopFrame, taskID) {
this._size = size;
this._buffer = {};
this._contextImage = {};
this._requestedChunks = {};
this._chunkSize = chunkSize;
this._stopFrame = stopFrame;
......@@ -348,6 +349,18 @@
this._taskID = taskID;
}
isContextImageAvailable(frame) {
return frame in this._contextImage;
}
getContextImage(frame) {
return this._contextImage[frame] || null;
}
addContextImage(frame, data) {
this._contextImage[frame] = data;
}
getFreeBufferSize() {
let requestedFrameCount = 0;
for (const chunk of Object.values(this._requestedChunks)) {
......@@ -535,6 +548,37 @@
}
}
async function getImageContext(taskID, frame) {
return new Promise((resolve, reject) => {
serverProxy.frames
.getImageContext(taskID, frame)
.then((result) => {
if (isNode) {
// eslint-disable-next-line no-undef
resolve(global.Buffer.from(result, 'binary').toString('base64'));
} else if (isBrowser) {
const reader = new FileReader();
reader.onload = () => {
resolve(reader.result);
};
reader.readAsDataURL(result);
}
})
.catch((error) => {
reject(error);
});
});
}
async function getContextImage(taskID, frame) {
if (frameDataCache[taskID].frameBuffer.isContextImageAvailable(frame)) {
return frameDataCache[taskID].frameBuffer.getContextImage(frame);
}
const response = getImageContext(taskID, frame);
frameDataCache[taskID].frameBuffer.addContextImage(frame, response);
return frameDataCache[taskID].frameBuffer.getContextImage(frame);
}
async function getPreview(taskID) {
return new Promise((resolve, reject) => {
// Just go to server and get preview (no any cache)
......@@ -558,7 +602,18 @@
});
}
async function getFrame(taskID, chunkSize, chunkType, mode, frame, startFrame, stopFrame, isPlaying, step) {
async function getFrame(
taskID,
chunkSize,
chunkType,
mode,
frame,
startFrame,
stopFrame,
isPlaying,
step,
dimension,
) {
if (!(taskID in frameDataCache)) {
const blockType = chunkType === 'video' ? cvatData.BlockType.MP4VIDEO : cvatData.BlockType.ARCHIVE;
......@@ -584,6 +639,7 @@
Math.max(decodedBlocksCacheSize, 9),
decodedBlocksCacheSize,
1,
dimension,
),
frameBuffer: new FrameBuffer(
Math.min(180, decodedBlocksCacheSize * chunkSize),
......@@ -630,5 +686,6 @@
getRanges,
getPreview,
clear,
getContextImage,
};
})();
......@@ -718,6 +718,29 @@
return response.data;
}
async function getImageContext(tid, frame) {
const { backendAPI } = config;
let response = null;
try {
response = await Axios.get(
`${backendAPI}/tasks/${tid}/data?quality=original&type=context_image&number=${frame}`,
{
proxy: config.proxy,
responseType: 'blob',
},
);
} catch (errorData) {
const code = errorData.response ? errorData.response.status : errorData.code;
throw new ServerError(
`Could not get Image Context of the frame for the task ${tid} from the server`,
code,
);
}
return response.data;
}
async function getData(tid, chunk) {
const { backendAPI } = config;
......@@ -1053,6 +1076,7 @@
getData,
getMeta,
getPreview,
getImageContext,
}),
writable: false,
},
......
......@@ -8,7 +8,7 @@
const loggerStorage = require('./logger-storage');
const serverProxy = require('./server-proxy');
const {
getFrame, getRanges, getPreview, clear: clearFrames,
getFrame, getRanges, getPreview, clear: clearFrames, getContextImage,
} = require('./frames');
const { ArgumentError } = require('./exceptions');
const { TaskStatus } = require('./enums');
......@@ -183,6 +183,15 @@
const result = await PluginRegistry.apiWrapper.call(this, prototype.frames.preview);
return result;
},
async contextImage(taskId, frameId) {
const result = await PluginRegistry.apiWrapper.call(
this,
prototype.frames.contextImage,
taskId,
frameId,
);
return result;
},
},
writable: true,
}),
......@@ -850,6 +859,7 @@
get: Object.getPrototypeOf(this).frames.get.bind(this),
ranges: Object.getPrototypeOf(this).frames.ranges.bind(this),
preview: Object.getPrototypeOf(this).frames.preview.bind(this),
contextImage: Object.getPrototypeOf(this).frames.contextImage.bind(this),
};
this.logger = {
......@@ -1501,6 +1511,7 @@
get: Object.getPrototypeOf(this).frames.get.bind(this),
ranges: Object.getPrototypeOf(this).frames.ranges.bind(this),
preview: Object.getPrototypeOf(this).frames.preview.bind(this),
contextImage: Object.getPrototypeOf(this).frames.contextImage.bind(this),
};
this.logger = {
......@@ -1683,6 +1694,7 @@
this.stopFrame,
isPlaying,
step,
this.task.dimension,
);
return frameData;
};
......@@ -2142,4 +2154,9 @@
const result = await loggerStorage.log(logType, { ...payload, task_id: this.id }, wait);
return result;
};
Job.prototype.frames.contextImage.implementation = async function (taskId, frameId) {
const result = await getContextImage(taskId, frameId);
return result;
};
})();
// Copyright (C) 2020 Intel Corporation
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
......@@ -13,8 +13,20 @@ const BlockType = Object.freeze({
ARCHIVE: 'archive',
});
const DimensionType = Object.freeze({
DIM_3D: '3d',
DIM_2D: '2d',
});
class FrameProvider {
constructor(blockType, blockSize, cachedBlockCount, decodedBlocksCacheSize = 5, maxWorkerThreadCount = 2) {
constructor(
blockType,
blockSize,
cachedBlockCount,
decodedBlocksCacheSize = 5,
maxWorkerThreadCount = 2,
dimension = DimensionType.DIM_2D,
) {
this._frames = {};
this._cachedBlockCount = Math.max(1, cachedBlockCount); // number of stored blocks
this._decodedBlocksCacheSize = decodedBlocksCacheSize;
......@@ -33,6 +45,7 @@ class FrameProvider {
this._mutex = new Mutex();
this._promisedFrames = {};
this._maxWorkerThreadCount = maxWorkerThreadCount;
this._dimension = dimension;
}
async _worker() {
......@@ -291,7 +304,7 @@ class FrameProvider {
};
worker.onmessage = async (event) => {
if (event.data.isRaw) {
if (this._dimension === DimensionType.DIM_2D && event.data.isRaw) {
// safary doesn't support createImageBitmap
// there is a way to polyfill it with using document.createElement
// but document.createElement doesn't work in worker
......@@ -328,8 +341,14 @@ class FrameProvider {
}
index++;
};
worker.postMessage({ block, start, end });
const dimension = this._dimension;
worker.postMessage({
block,
start,
end,
dimension,
dimension2D: DimensionType.DIM_2D,
});
this._decodeThreadCount++;
}
} finally {
......@@ -357,4 +376,5 @@ class FrameProvider {
module.exports = {
FrameProvider,
BlockType,
DimensionType,
};
......@@ -7,7 +7,7 @@ const JSZip = require('jszip');
onmessage = (e) => {
const zip = new JSZip();
if (e.data) {
const { start, end, block } = e.data;
const { start, end, block, dimension, dimension2D } = e.data;
zip.loadAsync(block).then((_zip) => {
let index = start;
......@@ -18,7 +18,7 @@ onmessage = (e) => {
.async('blob')
.then((fileData) => {
// eslint-disable-next-line no-restricted-globals
if (self.createImageBitmap) {
if (dimension === dimension2D && self.createImageBitmap) {
createImageBitmap(fileData).then((img) => {
postMessage({
fileName: relativePath,
......
......@@ -3988,6 +3988,19 @@
"array-find-index": "^1.0.1"
}
},
"cvat-canvas3d": {
"version": "file:../cvat-canvas3d",
"requires": {
"three": "^0.125.0"
},
"dependencies": {
"three": {
"version": "0.125.2",
"resolved": "https://registry.npmjs.org/three/-/three-0.125.2.tgz",
"integrity": "sha512-7rIRO23jVKWcAPFdW/HREU2NZMGWPBZ4XwEMt0Ak0jwLUKVJhcKM55eCBWyGZq/KiQbeo1IeuAoo/9l2dzhTXA=="
}
}
},
"cvat-canvas": {
"version": "file:../cvat-canvas",
"requires": {
......@@ -60,6 +60,7 @@
"@types/redux-logger": "^3.0.8",
"antd": "^4.10.2",
"copy-to-clipboard": "^3.3.1",
"cvat-canvas3d": "file:../cvat-canvas3d",
"cvat-canvas": "file:../cvat-canvas",
"cvat-core": "file:../cvat-core",
"dotenv-webpack": "^1.8.0",
......
// Copyright (C) 2020 Intel Corporation
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
......@@ -18,6 +18,7 @@ import {
ContextMenuType,
Workspace,
Model,
DimensionType,
OpenCVTool,
} from 'reducers/interfaces';
......@@ -190,6 +191,8 @@ export enum AnnotationActionTypes {
SWITCH_REQUEST_REVIEW_DIALOG = 'SWITCH_REQUEST_REVIEW_DIALOG',
SWITCH_SUBMIT_REVIEW_DIALOG = 'SWITCH_SUBMIT_REVIEW_DIALOG',
SET_FORCE_EXIT_ANNOTATION_PAGE_FLAG = 'SET_FORCE_EXIT_ANNOTATION_PAGE_FLAG',
HIDE_SHOW_CONTEXT_IMAGE = 'HIDE_SHOW_CONTEXT_IMAGE',
GET_CONTEXT_IMAGE = 'GET_CONTEXT_IMAGE',
}
export function saveLogsAsync(): ThunkAction {
......@@ -958,6 +961,10 @@ export function getJobAsync(tid: number, jid: number, initialFrame: number, init
maxZ,
},
});
if (job.task.dimension === DimensionType.DIM_3D) {
const workspace = Workspace.STANDARD3D;
dispatch(changeWorkspace(workspace));
}
dispatch(changeFrameAsync(frameNumber, false));
} catch (error) {
dispatch({
......@@ -1522,3 +1529,46 @@ export function setForceExitAnnotationFlag(forceExit: boolean): AnyAction {
},
};
}
export function hideShowContextImage(hidden: boolean): AnyAction {
return {
type: AnnotationActionTypes.HIDE_SHOW_CONTEXT_IMAGE,
payload: {
hidden,
},
};
}
export function getContextImage(): ThunkAction {
return async (dispatch: ActionCreator<Dispatch>): Promise<void> => {
const state: CombinedState = getStore().getState();
const { instance: job } = state.annotation.job;
const { frame, contextImage } = state.annotation.player;
try {
const context = await job.frames.contextImage(job.task.id, frame.number);
const loaded = true;
const contextImageHide = contextImage.hidden;
dispatch({
type: AnnotationActionTypes.GET_CONTEXT_IMAGE,
payload: {
context,
loaded,
contextImageHide,
},
});
} catch (error) {
const context = '';
const loaded = true;
const contextImageHide = contextImage.hidden;
dispatch({
type: AnnotationActionTypes.GET_CONTEXT_IMAGE,
payload: {
context,
loaded,
contextImageHide,
},
});
}
};
}
......@@ -18,6 +18,7 @@ import TagAnnotationWorkspace from 'components/annotation-page/tag-annotation-wo
import ReviewAnnotationsWorkspace from 'components/annotation-page/review-workspace/review-workspace';
import SubmitAnnotationsModal from 'components/annotation-page/request-review-modal';
import SubmitReviewModal from 'components/annotation-page/review/submit-review-modal';
import StandardWorkspace3DComponent from 'components/annotation-page/standard3D-workspace/standard3D-workspace';
interface Props {
job: any | null | undefined;
......@@ -79,23 +80,28 @@ export default function AnnotationPageComponent(props: Props): JSX.Element {
<Layout.Header className='cvat-annotation-header'>
<AnnotationTopBarContainer />
</Layout.Header>
{workspace === Workspace.STANDARD3D && (
<Layout.Content className='cvat-annotation-layout-content'>
<StandardWorkspace3DComponent />
</Layout.Content>
)}
{workspace === Workspace.STANDARD && (
<Layout.Content style={{ height: '100%' }}>
<Layout.Content className='cvat-annotation-layout-content'>
<StandardWorkspaceComponent />
</Layout.Content>
)}
{workspace === Workspace.ATTRIBUTE_ANNOTATION && (
<Layout.Content style={{ height: '100%' }}>
<Layout.Content className='cvat-annotation-layout-content'>
<AttributeAnnotationWorkspace />
</Layout.Content>
)}
{workspace === Workspace.TAG_ANNOTATION && (
<Layout.Content style={{ height: '100%' }}>
<Layout.Content className='cvat-annotation-layout-content'>
<TagAnnotationWorkspace />
</Layout.Content>
)}
{workspace === Workspace.REVIEW_WORKSPACE && (
<Layout.Content style={{ height: '100%' }}>
<Layout.Content className='cvat-annotation-layout-content'>
<ReviewAnnotationsWorkspace />
</Layout.Content>
)}
......
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
import React, { ReactElement, useEffect, useRef } from 'react';
import { GlobalHotKeys } from 'react-hotkeys';
import Layout from 'antd/lib/layout/layout';
import { Workspace } from 'reducers/interfaces';
import { Canvas3d } from 'cvat-canvas3d-wrapper';
import ContextImage from '../standard3D-workspace/context-image/context-image';
interface Props {
canvasInstance: Canvas3d;
jobInstance: any;
frameData: any;
curZLayer: number;
contextImageHide: boolean;
loaded: boolean;
data: string;
annotations: any[];
onSetupCanvas: () => void;
getContextImage(): void;
workspace: Workspace;
animateID: any;
automaticBordering: boolean;
showObjectsTextAlways: boolean;
}
const CanvasWrapperComponent = (props: Props): ReactElement => {
const animateId = useRef(0);
const cvatCanvasContainerRef = useRef();
const {
frameData, contextImageHide, getContextImage, loaded, data, annotations, curZLayer,
} = props;
const fitCanvas = (): void => {
const { canvasInstance } = props;
canvasInstance.fitCanvas();
};
const onCanvasSetup = (): void => {
const { onSetupCanvas } = props;
onSetupCanvas();
};
const animateCanvas = (): void => {
const { canvasInstance } = props;
canvasInstance.render();
animateId.current = requestAnimationFrame(animateCanvas);
};
const updateCanvas = (): void => {
const { canvasInstance } = props;
if (frameData !== null) {
canvasInstance.setup(frameData);
}
};
const initialSetup = (): void => {
const { canvasInstance } = props;
// Size
window.addEventListener('resize', fitCanvas);
fitCanvas();
// Events
canvasInstance.html().addEventListener('canvas.setup', onCanvasSetup);
};
useEffect(() => {
const { canvasInstance } = props;
cvatCanvasContainerRef.current.appendChild(canvasInstance.html());
initialSetup();
updateCanvas();
animateCanvas();
return () => {
canvasInstance.html().removeEventListener('canvas.setup', onCanvasSetup);
window.removeEventListener('resize', fitCanvas);
cancelAnimationFrame(animateId.current);
};
});
useEffect(() => {
updateCanvas();
}, [frameData, annotations, curZLayer]);
return (
<Layout.Content style={{ position: 'relative' }}>
<GlobalHotKeys />
<ContextImage
frame={frameData}
contextImageHide={contextImageHide}
getContextImage={getContextImage}
loaded={loaded}
data={data}
/>
<div ref={cvatCanvasContainerRef} className='cvat-canvas-container cvat-canvas-container-overflow' />
</Layout.Content>
);
};
export default React.memo(CanvasWrapperComponent);
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
import React, { useEffect } from 'react';
interface Props {
frame: number;
contextImageHide: boolean;
loaded: boolean;
data: string;
getContextImage(): void;
}
export default function ContextImage(props: Props): JSX.Element {
const {
contextImageHide, loaded, data, getContextImage,
} = props;
useEffect(() => {
if (!contextImageHide && !loaded) {
getContextImage();
}
}, [contextImageHide, loaded]);
if (!contextImageHide && data !== '') {
return (
<div className='cvat-contextImage'>
<img src={data} alt='Context not available' className='cvat-contextImage-show' />
</div>
);
}
return null;
}
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
import React from 'react';
import { GlobalHotKeys } from 'react-hotkeys';
import Layout from 'antd/lib/layout';
import { ActiveControl } from 'reducers/interfaces';
import { Canvas3d as Canvas } from 'cvat-canvas3d-wrapper';
import CursorControl from './cursor-control';
import MoveControl from './move-control';
import DrawCuboidControl from './draw-cuboid-control';
import PhotoContextControl from './photo-context';
interface Props {
canvasInstance: Canvas;
activeControl: ActiveControl;
normalizedKeyMap: Record<string, string>;
contextImageHide: boolean;
hideShowContextImage: (hidden: boolean) => void;
}
export default function ControlsSideBarComponent(props: Props): JSX.Element {
const {
canvasInstance, activeControl, normalizedKeyMap, contextImageHide, hideShowContextImage,
} = props;
return (
<Layout.Sider className='cvat-canvas-controls-sidebar' theme='light' width={44}>
<GlobalHotKeys />
<MoveControl canvasInstance={canvasInstance} activeControl={activeControl} />
<CursorControl
cursorShortkey={normalizedKeyMap.CANCEL}
canvasInstance={canvasInstance}
activeControl={activeControl}
/>
<DrawCuboidControl
canvasInstance={canvasInstance}
isDrawing={activeControl === ActiveControl.DRAW_CUBOID}
/>
<PhotoContextControl
canvasInstance={canvasInstance}
activeControl={activeControl}
contextImageHide={contextImageHide}
hideShowContextImage={hideShowContextImage}
/>
</Layout.Sider>
);
}
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
import React from 'react';
import Icon from '@ant-design/icons';
import Tooltip from 'antd/lib/tooltip';
import { CursorIcon } from 'icons';
import { ActiveControl } from 'reducers/interfaces';
import { Canvas3d as Canvas } from 'cvat-canvas3d-wrapper';
interface Props {
canvasInstance: Canvas;
cursorShortkey: string;
activeControl: ActiveControl;
}
function CursorControl(props: Props): JSX.Element {
const { activeControl, cursorShortkey } = props;
return (
<Tooltip title={`Cursor ${cursorShortkey}`} placement='right' mouseLeaveDelay={0}>
<Icon
component={CursorIcon}
className={[
'cvat-cursor-control',
activeControl === ActiveControl.CURSOR ? 'cvat-active-canvas-control ' : null,
]}
/>
</Tooltip>
);
}
export default React.memo(CursorControl);
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
import React from 'react';
import Popover from 'antd/lib/popover';
import Icon from '@ant-design/icons';
import { Canvas3d as Canvas } from 'cvat-canvas3d-wrapper';
import { ShapeType } from 'reducers/interfaces';
import { CubeIcon } from 'icons';
import DrawShapePopoverContainer from 'containers/annotation-page/standard-workspace/controls-side-bar/draw-shape-popover';
interface Props {
canvasInstance: Canvas;
isDrawing: boolean;
}
function DrawPolygonControl(props: Props): JSX.Element {
const { canvasInstance, isDrawing } = props;
const dynamcPopoverPros = isDrawing ?
{
overlayStyle: {
display: 'none',
},
} :
{};
const dynamicIconProps = isDrawing ?
{
className: 'cvat-draw-cuboid-control cvat-active-canvas-control',
onClick: (): void => {
canvasInstance.draw({ enabled: false });
},
} :
{
className: 'cvat-draw-cuboid-control',
};
return (
<Popover
{...dynamcPopoverPros}
overlayClassName='cvat-draw-shape-popover'
placement='right'
content={<DrawShapePopoverContainer shapeType={ShapeType.CUBOID} />}
>
<Icon {...dynamicIconProps} component={CubeIcon} />
</Popover>
);
}
export default React.memo(DrawPolygonControl);
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
import React from 'react';
import Icon from '@ant-design/icons';
import Tooltip from 'antd/lib/tooltip';
import { MoveIcon } from 'icons';
import { ActiveControl } from 'reducers/interfaces';
import { Canvas3d as Canvas } from 'cvat-canvas3d-wrapper';
interface Props {
canvasInstance: Canvas;
activeControl: ActiveControl;
}
function MoveControl(props: Props): JSX.Element {
const { activeControl } = props;
return (
<Tooltip title='Move the image' placement='right' mouseLeaveDelay={0}>
<Icon
component={MoveIcon}
className={[
'cvat-move-control',
activeControl === ActiveControl.DRAG_CANVAS ? ' cvat-active-canvas-control' : null,
]}
/>
</Tooltip>
);
}
export default React.memo(MoveControl);
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
import CameraIcon from '@ant-design/icons/CameraOutlined';
import Tooltip from 'antd/lib/tooltip';
import { Canvas3d as Canvas } from 'cvat-canvas3d-wrapper';
import React from 'react';
import { ActiveControl } from 'reducers/interfaces';
interface Props {
canvasInstance: Canvas;
activeControl: ActiveControl;
hideShowContextImage: (hidden: boolean) => void;
contextImageHide: boolean;
}
function PhotoContextControl(props: Props): JSX.Element {
const { activeControl, contextImageHide, hideShowContextImage } = props;
return (
<Tooltip title='Photo context show/hide' placement='right' mouseLeaveDelay={0}>
<CameraIcon
className={`cvat-move-control
cvat-control-side-bar-icon-size ${
activeControl === ActiveControl.PHOTO_CONTEXT ? 'cvat-active-canvas-control' : ''
}`}
onClick={(): void => {
hideShowContextImage(!contextImageHide);
}}
/>
</Tooltip>
);
}
export default React.memo(PhotoContextControl);
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
import './styles.scss';
import React from 'react';
import Layout from 'antd/lib/layout';
import CanvasWrapperContainer from 'containers/annotation-page/canvas/canvas-wrapper3D';
import ControlsSideBarContainer from 'containers/annotation-page/standard3D-workspace/controls-side-bar/controls-side-bar';
export default function StandardWorkspace3DComponent(): JSX.Element {
return (
<Layout hasSider className='cvat-standard-workspace'>
<ControlsSideBarContainer />
<CanvasWrapperContainer />
</Layout>
);
}
// Copyright (C) 2020 Intel Corporation
//
// SPDX-License-Identifier: MIT
@import 'base.scss';
.cvat-standard-workspace.ant-layout {
height: 100%;
}
.cvat-contextImage {
width: $grid-unit-size * 32;
position: absolute;
background: $border-color-3;
top: $grid-unit-size;
right: $grid-unit-size;
max-height: $grid-unit-size * 16;
z-index: 100;
border-radius: $grid-unit-size;
border: 1px solid $border-color-3;
display: flex;
flex-direction: column;
justify-content: space-between;
padding: $grid-unit-size/2;
}
.cvat-contextImage-show {
max-width: 100%;
max-height: 100%;
}
.cvat-contextImage-loading {
text-align: center;
}
.cvat-objects-sidebar-filter-input {
width: calc(100% - 35px);
}
.cvat-objects-sidebar-sider {
top: 0;
right: 0;
left: auto;
background-color: $background-color-2;
border-left: 1px solid $border-color-1;
border-bottom: 1px solid $border-color-1;
border-radius: $grid-unit-size/2 0 0 $grid-unit-size/2;
z-index: 2;
}
.cvat-objects-sidebar {
height: 100%;
}
.cvat-rotate-canvas-controls-right > svg {
transform: scaleX(-1);
}
.cvat-canvas-controls-sidebar {
background-color: $background-color-2;
border-right: 1px solid $border-color-1;
> div {
> i {
border-radius: 3.3px;
transform: scale(0.65);
padding: $grid-unit-size/4;
&:hover {
background: $header-color;
transform: scale(0.75);
}
&:active {
transform: scale(0.65);
}
> svg {
transform: scale(0.8);
}
}
}
}
.cvat-active-canvas-control {
background: $header-color;
transform: scale(0.75);
}
.cvat-rotate-canvas-controls-left,
.cvat-rotate-canvas-controls-right {
transform: scale(0.65);
border-radius: $grid-unit-size/2;
&:hover {
transform: scale(0.75);
}
&:active {
transform: scale(0.65);
}
}
.cvat-rotate-canvas-controls > .ant-popover-content > .ant-popover-inner > div > .ant-popover-inner-content {
padding: 0;
}
.cvat-draw-shape-popover,
.cvat-tools-control-popover {
> .ant-popover-content > .ant-popover-inner > div > .ant-popover-inner-content {
padding: 0;
}
}
.cvat-tools-track-button,
.cvat-tools-interact-button {
width: 100%;
margin-top: $grid-unit-size;
}
.cvat-draw-shape-popover-points-selector {
width: 100%;
}
.cvat-tools-control-popover-content {
width: fit-content;
padding: $grid-unit-size;
border-radius: $grid-unit-size/2;
background: $background-color-2;
}
.cvat-draw-shape-popover-content {
padding: $grid-unit-size;
border-radius: $grid-unit-size/2;
background: $background-color-2;
width: 270px;
> div {
margin-top: $grid-unit-size/2;
}
> div:nth-child(3) > div > div {
width: 100%;
}
> div:last-child {
span {
width: 100%;
}
button {
width: 100%;
&:nth-child(1) {
border-radius: $grid-unit-size/2 0 0 $grid-unit-size/2;
}
&:nth-child(2) {
border-radius: 0 $grid-unit-size/2 $grid-unit-size/2 0;
}
}
}
}
.cvat-canvas-container-overflow {
overflow: hidden;
width: 100%;
height: 100%;
}
.cvat-control-side-bar-icon-size {
font-size: $grid-unit-size * 5;
}
......@@ -9,6 +9,10 @@
overflow: hidden;
}
.cvat-annotation-layout-content {
height: 100%;
}
.ant-layout-header.cvat-annotation-header {
background-color: $background-color-2;
border-bottom: 1px solid $border-color-1;
......
......@@ -8,17 +8,20 @@ import Icon from '@ant-design/icons';
import Select from 'antd/lib/select';
import Button from 'antd/lib/button';
import { Workspace } from 'reducers/interfaces';
import { DimensionType, Workspace } from 'reducers/interfaces';
import { InfoIcon, FullscreenIcon } from 'icons';
interface Props {
workspace: Workspace;
showStatistics(): void;
changeWorkspace(workspace: Workspace): void;
jobInstance: any;
}
function RightGroup(props: Props): JSX.Element {
const { showStatistics, changeWorkspace, workspace } = props;
const {
showStatistics, changeWorkspace, workspace, jobInstance,
} = props;
return (
<Col className='cvat-annotation-header-right-group'>
......@@ -49,11 +52,26 @@ function RightGroup(props: Props): JSX.Element {
onChange={changeWorkspace}
value={workspace}
>
{Object.values(Workspace).map((ws) => (
<Select.Option key={ws} value={ws}>
{ws}
</Select.Option>
))}
{Object.values(Workspace).map((ws) => {
if (jobInstance.task.dimension === DimensionType.DIM_3D) {
if (ws === Workspace.STANDARD) {
return null;
}
return (
<Select.Option disabled={ws !== Workspace.STANDARD3D} key={ws} value={ws}>
{ws}
</Select.Option>
);
}
if (ws !== Workspace.STANDARD3D) {
return (
<Select.Option key={ws} value={ws}>
{ws}
</Select.Option>
);
}
return null;
})}
</Select>
</div>
</Col>
......
......@@ -53,6 +53,8 @@ interface Props {
onURLIconClick(): void;
onUndoClick(): void;
onRedoClick(): void;
jobInstance: any;
hideShowContextImage(): any;
}
export default function AnnotationTopBarComponent(props: Props): JSX.Element {
......@@ -96,6 +98,7 @@ export default function AnnotationTopBarComponent(props: Props): JSX.Element {
onURLIconClick,
onUndoClick,
onRedoClick,
jobInstance,
} = props;
return (
......@@ -146,7 +149,12 @@ export default function AnnotationTopBarComponent(props: Props): JSX.Element {
/>
</Row>
</Col>
<RightGroup workspace={workspace} changeWorkspace={changeWorkspace} showStatistics={showStatistics} />
<RightGroup
jobInstance={jobInstance}
workspace={workspace}
changeWorkspace={changeWorkspace}
showStatistics={showStatistics}
/>
</Row>
);
}
// Copyright (C) 2020 Intel Corporation
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
......
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
import { connect } from 'react-redux';
import CanvasWrapperComponent from 'components/annotation-page/canvas/canvas-wrapper3D';
import { confirmCanvasReady, getContextImage } from 'actions/annotation-actions';
import { CombinedState } from 'reducers/interfaces';
import { Canvas3d } from 'cvat-canvas3d-wrapper';
interface StateToProps {
canvasInstance: Canvas3d;
jobInstance: any;
frameData: any;
curZLayer: number;
contextImageHide: boolean;
loaded: boolean;
data: string;
annotations: any[];
}
interface DispatchToProps {
onSetupCanvas(): void;
getContextImage(): void;
}
function mapStateToProps(state: CombinedState): StateToProps {
const {
annotation: {
canvas: { instance: canvasInstance },
job: { instance: jobInstance },
player: {
frame: { data: frameData },
contextImage: { hidden: contextImageHide, data, loaded },
},
annotations: {
states: annotations,
zLayer: { cur: curZLayer },
},
},
} = state;
return {
canvasInstance,
jobInstance,
frameData,
curZLayer,
contextImageHide,
loaded,
data,
annotations,
};
}
function mapDispatchToProps(dispatch: any): DispatchToProps {
return {
onSetupCanvas(): void {
dispatch(confirmCanvasReady());
},
getContextImage(): void {
dispatch(getContextImage());
},
};
}
export default connect(mapStateToProps, mapDispatchToProps)(CanvasWrapperComponent);
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
import { ExtendedKeyMapOptions } from 'react-hotkeys';
import { connect } from 'react-redux';
import { Canvas } from 'cvat-canvas-wrapper';
import { hideShowContextImage } from 'actions/annotation-actions';
import ControlsSideBarComponent from 'components/annotation-page/standard3D-workspace/controls-side-bar/controls-side-bar';
import { ActiveControl, CombinedState } from 'reducers/interfaces';
interface StateToProps {
canvasInstance: Canvas;
activeControl: ActiveControl;
keyMap: Record<string, ExtendedKeyMapOptions>;
normalizedKeyMap: Record<string, string>;
contextImageHide: boolean;
loaded: boolean;
}
interface DispatchToProps {
hideShowContextImage(hidden: boolean): void;
}
function mapStateToProps(state: CombinedState): StateToProps {
const {
annotation: {
canvas: { instance: canvasInstance, activeControl },
player: {
contextImage: { hidden: contextImageHide, loaded },
},
},
shortcuts: { keyMap, normalizedKeyMap },
} = state;
return {
canvasInstance,
activeControl,
normalizedKeyMap,
keyMap,
contextImageHide,
loaded,
};
}
function dispatchToProps(dispatch: any): DispatchToProps {
return {
hideShowContextImage(hidden: boolean): void {
dispatch(hideShowContextImage(hidden));
},
};
}
export default connect(mapStateToProps, dispatchToProps)(ControlsSideBarComponent);
......@@ -625,6 +625,7 @@ class AnnotationTopBarContainer extends React.PureComponent<Props, State> {
focusFrameInputShortcut={normalizedKeyMap.FOCUS_INPUT_FRAME}
onUndoClick={this.undo}
onRedoClick={this.redo}
jobInstance={jobInstance}
/>
</>
);
......
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
import { Canvas3d, Canvas3dVersion } from 'cvat-canvas3d/src/typescript/canvas3d';
export { Canvas3d, Canvas3dVersion };
// Copyright (C) 2020 Intel Corporation
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT
......@@ -6,6 +6,7 @@ import React from 'react';
import { AnyAction } from 'redux';
import { Canvas, CanvasMode } from 'cvat-canvas-wrapper';
import { Canvas3d } from 'cvat-canvas3d-wrapper';
import { AnnotationActionTypes } from 'actions/annotation-actions';
import { AuthActionTypes } from 'actions/auth-actions';
import { BoundariesActionTypes } from 'actions/boundaries-actions';
......@@ -17,6 +18,7 @@ import {
ContextMenuType,
Workspace,
TaskStatus,
DimensionType,
} from './interfaces';
const defaultState: AnnotationState = {
......@@ -55,6 +57,11 @@ const defaultState: AnnotationState = {
},
playing: false,
frameAngles: [],
contextImage: {
loaded: false,
data: '',
hidden: false,
},
},
drawing: {
activeShapeType: ShapeType.RECTANGLE,
......@@ -133,7 +140,11 @@ export default (state = defaultState, action: AnyAction): AnnotationState => {
} = action.payload;
const isReview = job.status === TaskStatus.REVIEW;
let workspaceSelected = Workspace.STANDARD;
if (job.task.dimension === DimensionType.DIM_3D) {
workspaceSelected = Workspace.STANDARD3D;
}
return {
...state,
job: {
......@@ -176,10 +187,10 @@ export default (state = defaultState, action: AnyAction): AnnotationState => {
},
canvas: {
...state.canvas,
instance: new Canvas(),
instance: job.task.dimension === DimensionType.DIM_2D ? new Canvas() : new Canvas3d(),
},
colors,
workspace: isReview ? Workspace.REVIEW_WORKSPACE : Workspace.STANDARD,
workspace: isReview ? Workspace.REVIEW_WORKSPACE : workspaceSelected,
};
}
case AnnotationActionTypes.GET_JOB_FAILED: {
......@@ -201,6 +212,11 @@ export default (state = defaultState, action: AnyAction): AnnotationState => {
...state.player.frame,
fetching: false,
},
contextImage: {
loaded: false,
data: '',
hidden: state.player.contextImage.hidden,
},
},
};
}
......@@ -243,6 +259,10 @@ export default (state = defaultState, action: AnyAction): AnnotationState => {
changeTime,
delay,
},
contextImage: {
...state.player.contextImage,
loaded: false,
},
},
annotations: {
...state.annotations,
......@@ -1076,6 +1096,36 @@ export default (state = defaultState, action: AnyAction): AnnotationState => {
},
};
}
case AnnotationActionTypes.HIDE_SHOW_CONTEXT_IMAGE: {
const { hidden } = action.payload;
const { loaded, data } = state.player.contextImage;
return {
...state,
player: {
...state.player,
contextImage: {
loaded,
data,
hidden,
},
},
};
}
case AnnotationActionTypes.GET_CONTEXT_IMAGE: {
const { context, loaded } = action.payload;
return {
...state,
player: {
...state.player,
contextImage: {
loaded,
data: context,
hidden: state.player.contextImage.hidden,
},
},
};
}
case AnnotationActionTypes.CLOSE_JOB:
case AuthActionTypes.LOGOUT_SUCCESS: {
return { ...defaultState };
......
......@@ -6,6 +6,7 @@ import { ExtendedKeyMapOptions } from 'react-hotkeys';
import { Canvas, RectDrawingMethod } from 'cvat-canvas-wrapper';
import { IntelligentScissors } from 'utils/opencv-wrapper/intelligent-scissors';
import { MutableRefObject } from 'react';
import { Canvas3d } from 'cvat-canvas3d/src/typescript/canvas3d';
export type StringObject = {
[index: string]: string;
......@@ -332,6 +333,7 @@ export enum ActiveControl {
EDIT = 'edit',
OPEN_ISSUE = 'open_issue',
AI_TOOLS = 'ai_tools',
PHOTO_CONTEXT = 'PHOTO_CONTEXT',
OPENCV_TOOLS = 'opencv_tools',
}
......@@ -381,7 +383,7 @@ export interface AnnotationState {
pointID: number | null;
clientID: number | null;
};
instance: Canvas;
instance: Canvas | Canvas3d;
ready: boolean;
activeControl: ActiveControl;
};
......@@ -404,6 +406,11 @@ export interface AnnotationState {
};
playing: boolean;
frameAngles: number[];
contextImage: {
loaded: boolean;
data: string;
hidden: boolean;
};
};
drawing: {
activeInteractor?: Model | OpenCVTool;
......@@ -459,6 +466,7 @@ export interface AnnotationState {
}
export enum Workspace {
STANDARD3D = 'Standard 3D',
STANDARD = 'Standard',
ATTRIBUTE_ANNOTATION = 'Attribute annotation',
TAG_ANNOTATION = 'Tag annotation',
......
......@@ -4,11 +4,13 @@
import os
import os.path as osp
import io
import shutil
import traceback
from datetime import datetime
from distutils.util import strtobool
from tempfile import mkstemp
import cv2
import django_rq
from django.shortcuts import get_object_or_404
......@@ -39,7 +41,7 @@ from cvat.apps.dataset_manager.serializers import DatasetFormatsSerializer
from cvat.apps.engine.frame_provider import FrameProvider
from cvat.apps.engine.models import (
Job, StatusChoice, Task, Project, Review, Issue,
Comment, StorageMethodChoice, ReviewStatus, StorageChoice
Comment, StorageMethodChoice, ReviewStatus, StorageChoice, DimensionType, Image
)
from cvat.apps.engine.serializers import (
AboutSerializer, AnnotationFileSerializer, BasicUserSerializer,
......@@ -391,7 +393,7 @@ class TaskViewSet(auth.TaskGetQuerySetMixin, viewsets.ModelViewSet):
@swagger_auto_schema(method='get', operation_summary='Method returns data for a specific task',
manual_parameters=[
openapi.Parameter('type', in_=openapi.IN_QUERY, required=True, type=openapi.TYPE_STRING,
enum=['chunk', 'frame', 'preview'],
enum=['chunk', 'frame', 'preview', 'context_image'],
description="Specifies the type of the requested data"),
openapi.Parameter('quality', in_=openapi.IN_QUERY, required=True, type=openapi.TYPE_STRING,
enum=['compressed', 'original'],
......@@ -430,7 +432,7 @@ class TaskViewSet(auth.TaskGetQuerySetMixin, viewsets.ModelViewSet):
data_id = request.query_params.get('number', None)
data_quality = request.query_params.get('quality', 'compressed')
possible_data_type_values = ('chunk', 'frame', 'preview')
possible_data_type_values = ('chunk', 'frame', 'preview', 'context_image')
possible_quality_values = ('compressed', 'original')
try:
......@@ -475,6 +477,23 @@ class TaskViewSet(auth.TaskGetQuerySetMixin, viewsets.ModelViewSet):
elif data_type == 'preview':
return sendfile(request, frame_provider.get_preview())
elif data_type == 'context_image':
if db_task.dimension == DimensionType.DIM_3D:
data_id = int(data_id)
image = Image.objects.get(data_id=db_task.data_id, frame=data_id)
for i in image.related_files.all():
path = os.path.realpath(str(i.path))
image = cv2.imread(path)
success, result = cv2.imencode('.JPEG', image)
if not success:
raise Exception("Failed to encode image to '%s' format" % (".jpeg"))
return HttpResponse(io.BytesIO(result.tobytes()), content_type="image/jpeg")
return Response(data='No context image related to the frame',
status=status.HTTP_404_NOT_FOUND)
else:
return Response(data='Only 3D tasks support context images',
status=status.HTTP_400_BAD_REQUEST)
else:
return Response(data='unknown data type {}.'.format(data_type), status=status.HTTP_400_BAD_REQUEST)
except APIException as e:
......
......@@ -22,6 +22,7 @@ module.exports = (stagedFiles) => {
const cvatData = containsInPath('/cvat-data/', eslintFiles);
const cvatCore = containsInPath('/cvat-core/', eslintFiles);
const cvatCanvas = containsInPath('/cvat-canvas/', eslintFiles);
const cvatCanvas3d = containsInPath('/cvat-canvas3d/', eslintFiles);
const cvatUI = containsInPath('/cvat-ui/', eslintFiles);
const mapping = {};
......@@ -31,6 +32,7 @@ module.exports = (stagedFiles) => {
mapping['npm run precommit:cvat-data -- '] = cvatData.join(' ');
mapping['npm run precommit:cvat-core -- '] = cvatCore.join(' ');
mapping['npm run precommit:cvat-canvas -- '] = cvatCanvas.join(' ');
mapping['npm run precommit:cvat-canvas3d -- '] = cvatCanvas3d.join(' ');
for (const command of Object.keys(mapping)) {
const files = mapping[command];
......
......@@ -49,6 +49,7 @@
"precommit:cvat-data": "cd cvat-ui && eslint --fix",
"precommit:cvat-core": "cd cvat-ui && eslint --fix",
"precommit:cvat-canvas": "cd cvat-ui && eslint --fix",
"precommit:cvat-canvas3d": "cd cvat-ui && eslint --fix",
"precommit:cvat-ui": "cd cvat-ui && eslint --fix"
},
"repository": {
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册