* Improve rank inference for Expand op
Signed-off-by: NTom Wildenhain <tomwi@microsoft.com>

* Unnecessary dtype check in Expand shape inference
Signed-off-by: NTom Wildenhain <tomwi@microsoft.com>

* Add test for Expand rank inference
Signed-off-by: NTom Wildenhain <tomwi@microsoft.com>

* Update function defs to use parser
Signed-off-by: NGanesan Ramalingam <grama@microsoft.com>

* Extend parser to support attribute references
Signed-off-by: NGanesan Ramalingam <grama@microsoft.com>

* Update function body for MVN
Signed-off-by: NGanesan Ramalingam <grama@microsoft.com>

* Allow untyped inputs and outputs in graphs
Signed-off-by: NGanesan Ramalingam <grama@microsoft.com>

* Update range function body
Signed-off-by: NGanesan Ramalingam <grama@microsoft.com>

* Delete old function bodu builder for range
Signed-off-by: NGanesan Ramalingam <grama@microsoft.com>

* Minor formatting for range function body
Signed-off-by: NGanesan Ramalingam <grama@microsoft.com>

* Add support for comments in parser
Signed-off-by: NGanesan Ramalingam <grama@microsoft.com>

* Add support for exponent notation in float literals
Signed-off-by: NGanesan Ramalingam <grama@microsoft.com>

* Address PR feedback (add check for parse status)
Signed-off-by: NGanesan Ramalingam <grama@microsoft.com>

* Remove attr ref
Signed-off-by: NGanesan Ramalingam <grama@microsoft.com>

* Add back attr type in ref attr
Signed-off-by: NGanesan Ramalingam <grama@microsoft.com>
Co-authored-by: NAshwini Khade <askhade@microsoft.com>

* add more ops for data propagation
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* remove unused lib
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* improve nullptr for concat
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* fix && param warning
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* catch wrong broadcast
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* handle negative start and end for slice
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* refactor mathop
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* fixes by reviews
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* init nullptr
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* nit
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* consider symbolic shape. for math ops
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* ? as no value no param
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* nit comment typo
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* fix comments
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* spec clarification for matmulinteger and qlinearmatmul
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* update docs
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* fix typos
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* plus more typos
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* more typo fixes
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* crash fix
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* plus updates
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* Fix function body for SCE and NLL loss
Signed-off-by: Naishwarya bhandare <aibhanda@microsoft.com>

* review comments
Signed-off-by: Naishwarya bhandare <aibhanda@microsoft.com>

We don't enforce coercion of 2D anymore. Thus no need to mention 2D
specifically.
Signed-off-by: NZhenhua Wang <zhenhuaw@nvidia.com>
Co-authored-by: NG. Ramalingam <grama@microsoft.com>

* Add bfloat16 to Pow 2nd argument and bump to v15
Signed-off-by: NIan Bearman <ianb@microsoft.com>

* update docs
Signed-off-by: NIan Bearman <ianb@microsoft.com>
Co-authored-by: NG. Ramalingam <grama@microsoft.com>

Signed-off-by: NBowenBao <bowbao@microsoft.com>
Co-authored-by: NAshwini Khade <askhade@microsoft.com>

* Add HardSwish op.
Signed-off-by: NEwa21 <ewa.tusien@intel.com>

* Add docstring.
Signed-off-by: NEwa21 <ewa.tusien@intel.com>

* Undo unwanted changes.
Signed-off-by: NEwa21 <ewa.tusien@intel.com>

* Add type and shape inference.
Signed-off-by: NEwa21 <ewa.tusien@intel.com>

* Update docs.
Signed-off-by: NEwa21 <ewa.tusien@intel.com>

* Code formatting.
Signed-off-by: NEwa21 <ewa.tusien@intel.com>

* Update docs.
Signed-off-by: NEwa21 <ewa.tusien@intel.com>

* Change docstring and refactor code.
Signed-off-by: NEwa21 <ewa.tusien@intel.com>

* Add missing changes.
Signed-off-by: NEwa21 <ewa.tusien@intel.com>

* Add missing changes.
Signed-off-by: NEwa21 <ewa.tusien@intel.com>

* Bugfix
Signed-off-by: NEwa21 <ewa.tusien@intel.com>

* Update expanded model.
Signed-off-by: NEwa21 <ewa.tusien@intel.com>

* enable no exception build and update exception handling for data type utils
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* some fixes
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* fix CI failure
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* add ci pipeline for no exception builds
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* ci bug fix
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* fix shape inf failures
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* update CI name
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* plus review comments
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* test for windows ci
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* fixes for windows ci
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* fix typo
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* cmake fixes
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* fix linux build failure
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* fix mac build error
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* fix mac os warnings
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* some more mac warning fixes
Signed-off-by: NAshwini Khade <askhade@microsoft.com>

* add ALv2 to missed files and remove leftover MIT licenses
Signed-off-by: NSheng Zha <zhasheng@amazon.com>

* update proto files
Signed-off-by: NSheng Zha <zhasheng@amazon.com>

Signed-off-by: NPranav Prakash <pranav-prakash@users.noreply.github.com>

* [LICENSE] update license to Apache License v2.0
Signed-off-by: NSheng Zha <zhasheng@amazon.com>

* fix flake8
Signed-off-by: NSheng Zha <zhasheng@amazon.com>

* update auto generated docs
Signed-off-by: NSheng Zha <zhasheng@amazon.com>

* update cmake and bat
Signed-off-by: NSheng Zha <zhasheng@amazon.com>

* Update file to trigger DCO check
Signed-off-by: NPrasanth Pulavarthi <prasanth.pulavarthi@microsoft.com>

* CumSum-14: add input type float16 and bfloat16
Signed-off-by: NTianlei Wu <tlwu@microsoft.com>

* update data type comments
Signed-off-by: NTianlei Wu <tlwu@microsoft.com>

* Advance NLL loss to opset 13
Signed-off-by: NAishwarya <aibhanda@microsoft.com>

* Fix test for nll loss
Signed-off-by: NAishwarya <aibhanda@microsoft.com>
Co-authored-by: NAishwarya <aibhanda@OrtTrainingDev4.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>

* [Differentiability tags]
1. Revert Upsample's tags
2. Add some differentiable tags
Signed-off-by: NWei-Sheng Chin <wschin@outlook.com>

* Mark discontinous functions non-differentiable
Signed-off-by: NWei-Sheng Chin <wschin@outlook.com>

* Added const to a few Node methods
Signed-off-by: NAlexander Garustovich <yukigaru@gmail.com>

* Replaced .size() calls with .empty()
Signed-off-by: NAlexander Garustovich <yukigaru@gmail.com>

* Added a few reserve() for vectors, std::move and a few minor adjustments
Signed-off-by: NAlexander Garustovich <yukigaru@gmail.com>

* Updated according code review
Signed-off-by: NAlexander Garustovich <yukigaru@gmail.com>
Co-authored-by: NAshwini Khade <askhade@microsoft.com>

* Add differentiability indicators for math operators
Signed-off-by: NSvetlana Levitan <sveta_levitan@yahoo.com>

* Generate docs
Signed-off-by: NWei-Sheng Chin <wschin@outlook.com>

* Address comments....
Signed-off-by: NWei-Sheng Chin <wschin@outlook.com>
Co-authored-by: NWei-Sheng Chin <wschin@outlook.com>

* remove wrong description for pow
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* Add size check to make_tensor (#2987)
Co-authored-by: NKe Zhang <linkerzhang@yeah.net>
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* Fix float16 data convert issue in numpy_helper.to_array (#3002)

* handle f16 case for to_array

* fix flake8

* nit: comment
Co-authored-by: NAshwini Khade <askhade@microsoft.com>
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* Deprecate Travis CI (#2773)

* adding gtests for c++ api test

* deprecate travis

* adding the build badges for the new pipelines in Azure, deprecating travis build badge, renaming circleCI badge

* updating badge label

* removing - in badge names

* c++ api changes for linux

* update environment variables

* update env variables, setup tools call

* Update Linux-CI.yml for Azure Pipelines

* revert changes to Linux and Mac CIs

* delete last travis file
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* Fix shape inference of scalar ConstantOfShape (#3005)

When input shape is (0), we do not add any dim to inferred
shape but we should initialize tensor_type.shape by calling
mutable_shape().
Co-authored-by: NAshwini Khade <askhade@microsoft.com>
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* fix shape inference for loop (#3014)
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* Fix ConvTranspose: enhance attribute check (#3000)

* add check for using auto_pad and pads simultaneously

* fix description for auto_pads == SAME_UPPER

* update docs for operator

* fix the old one as well

* add a test

* Revert "fix description for auto_pads == SAME_UPPER"

This reverts commit e75e287f246c6fe7e2e54e0f2a73f15366cdad57.

* Revert "update docs for operator"

This reverts commit 70952c026c13153a04da0237033557c0eb9a14af.

* Revert "fix the old one as well"

This reverts commit 8a0482d3458636851380a495baad50d366a6325e.
Co-authored-by: NAshwini Khade <askhade@microsoft.com>
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* Schema change to support dynamic shapes in ORT (#2955)

* Changes to schema and python tests

* Modify test

* Remove attribute that is input also

* Changes to optimizers, adapters and tests

* Run flake8

* undo unrequired comit files, fix formatting, review changes

* Fix ci test, cleanup

* Fix narrowing conversion error

* add missed test model
Co-authored-by: NG. Ramalingam <grama@microsoft.com>
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* fix loop shape inference for ver 11 (#3023)
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>

* Update softmax family ops behavior to align with other frameworks (fix #2289) (#2879)

* Update softmax family ops behavior to align with other frameworks

* Update logsoftmax, hardmax tests, regenerate docs and test data

* fix wrong input name in function

* regenerate test data

* fix flake8 error

* regenerate docs

* regenerate docs

* add missing type annotation for hardmax

* add the math for softmax family operators

* remove the 'description' field in docs as it is covered by the math

* fix wrong format in axis attr

* replace name with description

* restore the name field for axis attr

* regenerate docs

* regenerate docs

* add the missing name

* regenerate docs

* update reducesum to align with master

* regenerate tests
Co-authored-by: NWei-Sheng Chin <wschin@outlook.com>
Signed-off-by: NChun-Wei Chen <jacky82226@gmail.com>
Co-authored-by: NG. Ramalingam <grama@microsoft.com>
Co-authored-by: NKe Zhang <linkerzhang@yeah.net>
Co-authored-by: NAshwini Khade <askhade@microsoft.com>
Co-authored-by: NVinitra Swamy <vinitras@gmail.com>
Co-authored-by: NShinichiro Hamaji <shinichiro.hamaji@gmail.com>
Co-authored-by: Nashbhandare <ash.bhandare@gmail.com>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: NWei-Sheng Chin <wschin@outlook.com>

* Update softmax family ops behavior to align with other frameworks

* Update logsoftmax, hardmax tests, regenerate docs and test data

* fix wrong input name in function

* regenerate test data

* fix flake8 error

* regenerate docs

* regenerate docs

* add missing type annotation for hardmax

* add the math for softmax family operators

* remove the 'description' field in docs as it is covered by the math

* fix wrong format in axis attr

* replace name with description

* restore the name field for axis attr

* regenerate docs

* regenerate docs

* add the missing name

* regenerate docs

* update reducesum to align with master

* regenerate tests
Co-authored-by: NWei-Sheng Chin <wschin@outlook.com>

* Changes to schema and python tests

* Modify test

* Remove attribute that is input also

* Changes to optimizers, adapters and tests

* Run flake8

* undo unrequired comit files, fix formatting, review changes

* Fix ci test, cleanup

* Fix narrowing conversion error

* add missed test model
Co-authored-by: NG. Ramalingam <grama@microsoft.com>

* add nullptr check to type inference mtds

* fix shape inference error

* plus review comments

* The axis input for CumSum is described as optional, however it was not denoted
as optional in the operator schema.
* Fixed test cases to pass a 0-D tensor for axis.
* Fixed 1-D reverse exclusive test case to set the exclusive attribute.
Signed-off-by: NJason Plurad <pluradj@us.ibm.com>
Co-authored-by: NG. Ramalingam <grama@microsoft.com>

* add bfloat16 to all numeric types

* add bfloat to more ops

* Bump versions

* add opset13 into supported range

* fix build err

* re-generate docs

* fix ci break

* add cast bfloat/float test cases

* remove useless code

* fix the export types

* fix the export types in script

* add doc

* fix the py27 error

* fix python style

* update docs

* add type in comments

* fix mypy error

* fix more mypy error

* bypass the unicode check
Co-authored-by: NCheng Tang <chenta@microsoft.com>
Co-authored-by: NKe Zhang <kezhan@microsoft.com>

* Revert "Fix for inverse spec (#2706)"

This reverts commit 1e887794.

* Revert "Adding Inverse Op (#2578)"

This reverts commit 8873cb02.

# Conflicts:
#	docs/Changelog.md
#	docs/TestCoverage.md
#	onnx/defs/math/defs.cc
#	onnx/defs/operator_sets.h

* Update MD files.

* Update MD files.

* merge conflicts.:

* regenerate MD files.

* Revert "Introduce MeanSquaredError as Loss Function (#2570)"

This reverts commit d2fe4f22.

# Conflicts:
#	docs/Changelog.md
#	docs/Operators.md
#	docs/TestCoverage.md
#	onnx/defs/math/defs.cc
#	onnx/defs/operator_sets.h

* Update MD files.

* update MD files.

* Trigger builds.

* Trigger builds.

* Add scaling factor to dropout test.

* Fix function definition for SoftmaxCrossEntropyLoss and Celu.

* Implement optional output log_probability in function definition and reference implementation for SotfmaxCrossEntropyLoss.

* Add shape inference for Celu.

* fix pep8 formating errors.

* build break.

* Handle int64 type target values in NegativeLogLossLikelihoodLoss function body.

* Fix shape inference for negativeloglikelihoodloss.

* Add training_mode to dropout spec and tests.

* fix build.

* Update dropout tests with seed=0.

* update old dropout data.

* Improve dropout test data.

* PR feedback.

* Remove stale dropout tests.

* PR feedback.

* update MD files.

* remove extra space.

* Updated shape inference to fix the bug for scalars

* Fix for formatting

* Added note in spec
Co-authored-by: NG. Ramalingam <grama@microsoft.com>

* Fixed spec for input rank

* Full-rank enforce

* Updated spec

* Updated spec for behavior

* Type constrain updated

* Refactor desc

* Shape inference changes

* Fail for invalid shapes

* float type only

* Typo

* Update SoftmaxCrossEntropyLoss and NegativeLogLikelihoodLoss documentation.

* Update documentation

* Update Operators.md

* Update SoftmaxCrossEntropyLoss and NegativeLogLikelihoodLoss documentation.

* Update SoftmaxCrossEntropyLoss and NegativeLogLikelihoodLoss documentation.
Co-authored-by: NG. Ramalingam <grama@microsoft.com>

Make NegativeLogLikelihoodLoss function body handle out of range ignore_index and re-enable all tests (#2703)

* Fix bug in NegativeLogLikelihood function body to handle out of range ignore_index and re-enable all tests.

* PR feedback.

Update `ignore_index` documentation for SoftmaxCrossEntropyLoss and NegativeLogLikelihoodLoss (#2696)

* Change type of label tensor to int32/int64 in SoftmaxCrossEntropyLoss spec.

* Update SoftmaxCrossEntropy and NLLLoss ignore_index doc.

* Update SoftmaxCrossEntropy and NLLLoss ignore_index doc.

* update doc.

* PR feedback.

* Fix for einsum

* Fix for ellipsis for diagonal

* Add check for rank

* Clean

* Implicit check added

* Change the return type for the zipmap operator to match the description in the spec.

* Reduce binary size of libraries consuming ONNX (part 1/2)

* Fix build error

* Replace separate Get*Doc() functions with easy macro for greater convenience

* Add one more macro for complicated operator doc documentation.
Co-authored-by: NKe Zhang <kezhan@microsoft.com>

* Add 'ignore_index' input in the spec for SoftmaxCrossEntropyLoss and NLLLoss.

* Add tests.

* build break.

* build break.

* clean up.

* build break.

* Change ignore_index to attribute.

* Change ignore_index to attribute.

* PR feedback.

* PR feedback.

* Make ignore_index optional in NLLLoss.

* Build break.

* remove trailing spaces to fix build break.

* Build break.

* Update spec doc.

* Fix NLLLoss function definition to fix test: test_negative_log_likelihood_loss_input_shape_is_NCd1d2_with_weight_reduction_sum_ignore_index_expanded

* PR feedback.

* Fix test for softmax cross entropy loss to exclude ignored_index'ed weights from the sum of weights.

* Build break.

* Update Pow input types in Opset 12

* gen doc and tests

* remove uints and 8 bit ints

* add tests

* remove uint int x tets