Neural Networks

Base

Functions

void hl_shrink_col2feature(const real *dataCol, size_t channels, size_t height, size_t width, size_t blockH, size_t blockW, size_t strideH, size_t strideW, size_t paddingH, size_t paddingW, size_t outputH, size_t outputW, real *dataIm, real alpha = 1.0f, real beta = 0.0f)

Shrink column to feature.

Parameters

• dataCol -

  expand data.

• channels -

  number of channel.

• height -

  image height.

• width -

  image width.

• blockH -

  filter height.

• blockW -

  filter width.

• strideH -

  stride height.

• strideW -

  stride width.

• paddingH -

  padding height.

• paddingW -

  padding width.

• outputH -

  output height.

• outputW -

  output width.

• dataIm -

  output image data.

• alpha -

• beta -

void hl_expand_feature2col(const real *dataIm, size_t channels, size_t height, size_t width, size_t blockH, size_t blockW, size_t strideH, size_t strideW, size_t paddingH, size_t paddingW, size_t outputH, size_t outputW, real *dataCol)

Expand feature to column.

Parameters

• dataIm -

  input image data.

• channels -

  number of channel.

• height -

  image height.

• width -

  image width.

• blockH -

  filter height.

• blockW -

  filter width.

• strideH -

  stride height.

• strideW -

  stride width.

• paddingH -

  padding height.

• paddingW -

  padding width.

• outputH -

  output height.

• outputW -

  output width.

• dataCol -

  expand data.

void hl_maxpool_forward(int frameCnt, const real *inputData, int channels, int height, int width, int pooledH, int pooledW, int sizeX, int stride, int start, real *tgtData)

Maximum pool forward.

Parameters

• frameCnt -

  batch size of input image.

• inputData -

  input data.

• channels -

  number of channel.

• height -

  image height.

• width -

  image width.

• pooledH -

  output image height.

• pooledW -

  output image width.

• sizeX -

  size of pooling window.

• stride -

  pooling stride.

• start -

  pooling start.

• tgtData -

  output data.

void hl_maxpool_backward(int frameCnt, const real *inputData, const real *outData, const real *outGrad, int channels, int height, int width, int pooledH, int pooledW, int sizeX, int stride, int start, real *targetGrad, real scaleA, real scaleB)

Maximum pool backward.

Parameters

• frameCnt -

  batch size of input image.

• inputData -

  input data.

• outData -

  output data.

• outGrad -

  output grad data.

• channels -

  number of channel.

• height -

  image height.

• width -

  image width.

• pooledH -

  output image height.

• pooledW -

  output image width.

• sizeX -

  size of pooling window.

• stride -

  pooling stride.

• start -

  pooling start.

• targetGrad -

  output grad.

• scaleA -

  scale.

• scaleB -

  scale.

void hl_avgpool_forward(int frameCnt, const real *inputData, int channels, int height, int width, int pooledH, int pooledW, int sizeX, int stride, int start, real *tgtData)

Averge pool forward.

Parameters

• frameCnt -

  batch size of input image.

• inputData -

  input data.

• channels -

  number of channel.

• height -

  image height.

• width -

  image width.

• pooledH -

  output image height.

• pooledW -

  output image width.

• sizeX -

  size of pooling window.

• stride -

  pooling stride.

• start -

  pooling start.

• tgtData -

  output data.

void hl_avgpool_backward(int frameCnt, const real *outGrad, int channels, int height, int width, int pooledH, int pooledW, int sizeX, int stride, int start, real *backGrad, real scaleA, real scaleB)

Maximum pool backward.

Parameters

• frameCnt -

  batch size of input image.

• outGrad -

  input data.

• channels -

  number of channel.

• height -

  image height.

• width -

  image width.

• pooledH -

  output image height.

• pooledW -

  output image width.

• sizeX -

  size of pooling window.

• stride -

  pooling stride.

• start -

  pooling start.

• backGrad -

  output grad.

• scaleA -

  scale.

• scaleB -

  scale.

void hl_CMRNorm_forward(size_t frameCnt, const real *in, real *scale, real *out, size_t channels, size_t height, size_t width, size_t sizeX, real alpha, real beta)

Cross-map-respose normalize forward.

Parameters

• frameCnt -

  batch size of input image.

• in -

  input data.

• scale -

  buffer.

• out -

  output data.

• channels -

  number of channel.

• height -

  image height.

• width -

  image width.

• sizeX -

  size.

• alpha -

  scale.

• beta -

  scale.

void hl_CMRNorm_backward(size_t frameCnt, const real *inV, const real *scale, const real *outV, const real *outDiff, real *inDiff, size_t channels, size_t height, size_t width, size_t sizeX, real alpha, real beta)

Cross-map-respose normalize backward.

Parameters

• frameCnt -

  batch size of input image.

• inV -

  input data.

• scale -

  buffer.

• outV -

  output value.

• outDiff -

  output grad.

• inDiff -

  input grad.

• channels -

  number of channel.

• height -

  image height.

• width -

  image width.

• sizeX -

  size.

• alpha -

  scale.

• beta -

  scale.

Defines

SIGMOID_THRESHOLD_MIN

sigmoid threshold maximum

SIGMOID_THRESHOLD_MAX

sigmoid threshold minimum

namespace hppl

Functions

real relu(const real a)

real sigmoid(const real a)

real tanh(const real a)

real linear(const real a)

real relu(const real a, const real b)

real sigmoid(const real a, const real b)

real tanh(const real a, const real b)

real linear(const real a, const real b)

namespace hppl

Functions

__m256 relu(const __m256 a)

__m256 sigmoid(const __m256 a)

__m256 tanh(const __m256 a)

__m256 linear(const __m256 a)

__m256 relu(const __m256 a, const __m256 b)

__m256 sigmoid(const __m256 a, const __m256 b)

__m256 tanh(const __m256 a, const __m256 b)

__m256 linear(const __m256 a, const __m256 b)

Activation Functions

Defines

HPPL_ACTIVE_FUNCTION

Active functions: sigmoid, relu, tanh and linear.

namespace hppl

template <class T>

class Active

#include <hl_activation_functions.h>

Hppl supports sigmoid, relu, tanh, linear active functions for neural networks’ forward and backward activation.

Public Types

typedef T (*forward)(T)

typedef T (*backward)(T, T)

namespace cpu

Variables

Active<real>::forward forward[]

Active<real>::backward backward[]

RNN Related APIs

Functions

void hl_max_sequence_forward(real *input, const int *sequence, real *output, int *index, int numSequences, int dim)

Maximum sequence forward.

Parameters

• input -

  each sequence contains some instances.

• sequence -

  sequence index..

• output -

  max instance in this sequence.

• index -

  index of max instance.

• numSequences -

  size of sequence[in].

• dim -

  input dimension.

void hl_max_sequence_backward(real *outputGrad, int *index, real *inputGrad, int numSequences, int dim)

Maximum sequence backward.

Parameters

• outputGrad -

  output gradient.

• index -

  index of max instance.

• inputGrad -

  input gradient.

• numSequences -

  size of sequence[in].

• dim -

  input dimension.

void hl_context_projection_forward(real *input, const int *sequence, real *weightData, real *output, int numSequences, int inputDim, int contextLength, int contextStart, int beginPad, bool isPadding)

Context projection forward.

Parameters

• input -

  input sequence.

• sequence -

  sequence index.

• weightData -

  padding data.

• output -

  output sequence.

• numSequences -

  number of sequences.

• inputDim -

  input sequence dimension.

• contextLength -

  context length.

• contextStart -

  context start.

• beginPad -

  number of extra timesteps added at the beginning.

• isPadding -

  trainable padding.

void hl_context_projection_backward_data(real *outputGrad, const int *sequence, real *inputGrad, int numSequences, int inputDim, int contextLength, int contextStart)

Context projection backward data.

Parameters

• outputGrad -

  output gradient.

• sequence -

  sequence index.

• inputGrad -

  input gradient.

• numSequences -

  number of sequences.

• inputDim -

  input sequence dimension.

• contextLength -

  context length.

• contextStart -

  context start.

void hl_context_projection_backward_weight(real *outputGrad, const int *sequence, real *weightGrad, int numSequences, int weightDim, int totalPad, int contextLength, int contextStart, int beginPad)

Context projection backward weight.

Parameters

• outputGrad -

  output gradient.

• sequence -

  sequence index.

• weightGrad -

  weight gradient.

• numSequences -

  number of sequences.

• weightDim -

  input sequence dimension.

• totalPad -

  number of extra timesteps.

• contextLength -

  context length.

• contextStart -

  context start.

• beginPad -

  number of extra timesteps added at the beginning.

void hl_sequence2batch_copy(real *batch, real *sequence, int *batchIndex, int seqWidth, int batchCount, bool seq2batch)

Memory copy from sequence to batch.

if seq2batch == true

copy from sequence to batch: batch[i] = sequence[batchIndex[i]].

if seq2batch == false

copy from batch to sequence: sequence[batchIndex[i]] = batch[i].

Parameters

• batch -

  batch matrix.

• sequence -

  equence matrix.

• batchIndex -

  index vector.

• seqWidth -

  width of sequence.

• batchCount -

  number of batchIndex.

• seq2batch -

  copy direction.

void hl_sequence2batch_add(real *batch, real *sequence, int *batchIndex, int seqWidth, int batchCount, bool seq2batch)

Add sequence to batch.

if seq2batch == true

add sequence to batch: batch[i] = sequence[batchIndex[i]].

if seq2batch == false

add batch to sequence: sequence[batchIndex[i]] = batch[i].

Parameters

• batch -

  batch matrix.

• sequence -

  equence matrix.

• batchIndex -

  index vector.

• seqWidth -

  width of sequence.

• batchCount -

  number of batchIndex.

• seq2batch -

  copy direction.

void hl_sequence_avg_forward(real *dst, real *src, const int *starts, int height, int width, const int mode)

dst = Op(src), src is sequence.

mode = 0, Op is average.

mode = 1, Op is sum.

mode = 2, Op is sum(src)/sqrt(N), N is sequence length.

Parameters

• dst -

  destination data.

• src -

  source data.

• starts -

  sequence start positions.

• height -

  height of dst data.

• width -

  width of dst data.

• mode -

  0: avreage, 1: sum, 2: divide by square root of sequenceLength

LSTM Model

Functions

void hl_lstm_parallel_forward(real *gateValue, real *stateValue, real *preOutputValue, real *outputValue, real *checkIg, real *checkFg, real *checkOg, real *weight, const int *sequence, int frameSize, int numSequences, bool reversed, hl_activation_mode_t active_node, hl_activation_mode_t active_gate, hl_activation_mode_t active_state)

Lstm sequence parallel forward.

Note

Only support frameSize = 32 or 64.

Parameters

• gateValue -

  input value.

• stateValue -

  state value.

• preOutputValue -

  prev output value.

• outputValue -

  output value.

• checkIg -

  bias.

• checkFg -

  bias.

• checkOg -

  bias.

• weight -

  weight.

• sequence -

  sequence index.

• frameSize -

  frame size.

• numSequences -

  number of sequences.

• reversed -

  reverse.

• active_node -

  active input type.

• active_gate -

  active state type.

• active_state -

  actvie gate type.

void hl_lstm_parallel_backward_data(real *gateValue, real *gateGrad, real *stateValue, real *stateGrad, real *preOutputValue, real *preOutputGrad, real *outputGrad, real *checkIg, real *checkIgGrad, real *checkFg, real *checkFgGrad, real *checkOg, real *checkOgGrad, real *weight, const int *sequence, int frameSize, int numSequences, bool reversed, hl_activation_mode_t active_node, hl_activation_mode_t active_gate, hl_activation_mode_t active_state)

Lstm sequence parallel backward data.

Note

Only support frameSize = 32 or 64.

Parameters

• gateValue -

  input value.

• gateGrad -

  input gradient.

• stateValue -

  state value.

• stateGrad -

  state gradient.

• preOutputValue -

  prev output value.

• preOutputGrad -

  prev output gradient.

• outputGrad -

  output gradient.

• checkIg -

  bias.

• checkIgGrad -

  bias gradient.

• checkFg -

  bias.

• checkFgGrad -

  bias gradient.

• checkOg -

  bias.

• checkOgGrad -

  bias gradient.

• weight -

  weight.

• sequence -

  sequence index.

• frameSize -

  frame size.

• numSequences -

  number of sequences.

• reversed -

  reverse.

• active_node -

  active input type.

• active_gate -

  active state type.

• active_state -

  actvie gate type.

void hl_lstm_parallel_backward_weight(real *weightGrad, real *outputValue, real *gateGrad, const int *sequence, int frameSize, int batchSize, int numSequences, bool reversed)

Lstm sequence parallel backward weight.

Parameters

• weightGrad -

  weight gradient.

• outputValue -

  output value.

• gateGrad -

  gate gradient.

• sequence -

  sequence index.

• frameSize -

  frame size.

• batchSize -

  batch size.

• numSequences -

  number of sequences.

• reversed -

  reverse.

GRU Model