- 15 8月, 2023 1 次提交
-
-
由 yinwei 提交于
--------- Co-authored-by: Ntianhaodongbd <tianhaodong@baidu.com>
-
- 07 8月, 2023 2 次提交
-
-
由 yin wei 提交于
* add mask * add backword * add enforce info * update scale * integrate code * update enforce * add enforce eq * add error type * update enforce * add test_flash_attention * Polish codes and fix compiling errors. * Set num_splits to 0 for flash-attn with tensor mask. * Fix the compiling error for non flash-attn case. --------- Co-authored-by: NLiu Yiqun <liuyiqun01@baidu.com>
-
由 umiswing 提交于
* Work for fa-2 padded fwd. Code to be cleaned. * Work for fa2 unpadded fwd. * Work for padded-bwd, dk get small diff on np.random.seed(0) * Anyway I pass paddle's utest, except return softmax without dropout. * Clean code. * Modify interface. * Clean code and add some check. * Easy compile for dev. * Fix ci. * Fix ci-build. * Add std c++17 option again. * Limit max job when compiling fa2. * Remove const_cast * Add fwd params, to be cleaned. * Clean code. * Add bwd params. * Clean code. * Add enforce. * Use v2.0.4 * Pass RNG state to fa2 capi * Fix review. * Add assert * Skip compile for sm less than 80.
-
- 20 4月, 2023 1 次提交
-
-
由 Chitsing KUI 提交于
* add flash randomness control * fix VLOG undefied
-
- 06 4月, 2023 1 次提交
-
-
由 Chitsing KUI 提交于
-
- 21 3月, 2023 1 次提交
-
-
由 ShenLiang 提交于
* fix flash_attention * Update mp_layers.py
-
- 16 3月, 2023 1 次提交
-
-
由 Chitsing KUI 提交于
* rename flash_attn_raw to flash_attn_unpadded * fix static api * fix static return
-
- 10 3月, 2023 1 次提交
-
-
由 Chitsing KUI 提交于
-
- 01 3月, 2023 1 次提交
-
-
由 Chitsing KUI 提交于
* flash attn * seed * almost * softmax * fix workspace * add unitest; linux only * fix setup * fix datatype include * fix setup typo * fix def scope * new error api * use paddle fork * fix attr bug; complete ut * update flash hash * fix rng reset * fix offset * fix comments
-