Commits

Commits on Dec 7, 2022

Fix Slurm Runner (#41)
Show description for 95a460d
dashstander
authored
95a460d
Copy full SHA for 95a460d

Commits on Sep 22, 2022

Multi-Node SLURM Support (#40)
Show description for e650c52

dashstander
and
shiv
authored
e650c52
Copy full SHA for e650c52

Commits on Sep 21, 2022

Update CODEOWNERS
StellaAthena
authored
4ae716d
Copy full SHA for 4ae716d

Commits on Apr 14, 2022

Only load optimizer states if the flag is set (#37)
Show description for 1726d1d
AeroScripts
authored
1726d1d
Copy full SHA for 1726d1d

Commits on Sep 20, 2021

fix bug with inputs requiring gradient
sdtblck
committed
eb7f5cf
Copy full SHA for eb7f5cf

Commits on Sep 17, 2021

refresh fp32 params from fp16 model weights if refreshing from state dict fails
sdtblck
committed
d9846af
Copy full SHA for d9846af

Commits on Sep 6, 2021

Move hooked embeddings to cpu to avoid oom (#34)
Show description for 95a5f00

sweinbach
and
Samuel Weinbach
authored
95a5f00
Copy full SHA for 95a5f00
Revert "make fp32 allreduce optional"
Show description for c745f3d
sdtblck
committed
c745f3d
Copy full SHA for c745f3d

Commits on Jul 29, 2021

make fp32 allreduce optional
sdtblck
committed
027485a
Copy full SHA for 027485a
fix large checkpoints in pipe parallel (#33)
Show description for 87fbb8f

sweinbach
and
Samuel Weinbach
authored
87fbb8f
Copy full SHA for 87fbb8f

Commits on Jun 8, 2021

update forward hook fn
sdtblck
authored
c45ec1c
Copy full SHA for c45ec1c

Commits on Jun 7, 2021

add forward hook functionality
server-jack
committed
8e6a834
Copy full SHA for 8e6a834
remove compressed allreduce
server-jack
committed
8900aa0
Copy full SHA for 8900aa0
Update requirements-sparse_attn.txt
sdtblck
authored
cfe1bab
Copy full SHA for cfe1bab

Commits on May 19, 2021

make checkpointable layers a list
sdtblck
authored
2d6a0d3
Copy full SHA for 2d6a0d3
add 'set_checkpointable_layers' func
sdtblck
authored
2d78b37
Copy full SHA for 2d78b37
Update engine.py
sdtblck
authored
faae906
Copy full SHA for faae906
make attention mask + pipe parallel more flexible
sdtblck
authored
3b49e7f
Copy full SHA for 3b49e7f
update 'is_checkpointable'
sdtblck
authored
9a384c9
Copy full SHA for 9a384c9

Commits on May 17, 2021

Update compressed_ar.py
sdtblck
authored
b467d11
Copy full SHA for b467d11

Commits on May 14, 2021

Bf16 (#31)
Show description for 996951e
sdtblck
authored
996951e
Copy full SHA for 996951e

Commits on May 11, 2021

add ability to return logits in eval
sdtblck
committed
3389e4f
Copy full SHA for 3389e4f

Commits on May 10, 2021

revert
sdtblck
authored
c7c2063
Copy full SHA for c7c2063
Update engine.py
sdtblck
authored
c814fca
Copy full SHA for c814fca

Commits on May 6, 2021

Update requirements-sparse_attn.txt
sdtblck
authored
75de46c
Copy full SHA for 75de46c

Commits on May 5, 2021

fix check for apex FusedAdam
sdtblck
authored
05a32ca
Copy full SHA for 05a32ca
fix check for apex fused adam
sdtblck
authored
cea5de1
Copy full SHA for cea5de1
Update requirements-sparse_attn.txt
sdtblck
authored
0959801
Copy full SHA for 0959801
Update requirements-sparse_attn.txt
sdtblck
authored
5614823
Copy full SHA for 5614823

Commits on May 4, 2021

If you try to reduce the loss lower than the loss scale - loss scaler will raise an error
sdtblck
committed
a6748a4
Copy full SHA for a6748a4

Commits on May 2, 2021

Add 'unidirectional' option to BigBird + BSLongformer Sparsity Configs
sdtblck
authored
24026e5
Copy full SHA for 24026e5
Update sparsity_config.py
sdtblck
authored
353387d
Copy full SHA for 353387d
Update __init__.py
sdtblck
authored
e4689e2
Copy full SHA for e4689e2
Update sparsity_config.py
sdtblck
authored
3d59d19
Copy full SHA for 3d59d19

Commits on May 1, 2021

Add `Local Sliding Window` Sparsity config
sdtblck
authored
73dfe6b
Copy full SHA for 73dfe6b