Mostofa Patwary

Mostofa Patwary

nVidia Corporation - / United States
Engineering & Technology / Computer Science
AD Scientific Index ID: 4414240
person_outline Mostofa Patwary's MOST POPULAR ARTICLES
1-) Megatron-LM: Training Multi-Billion Parameter Language Models Using GPU Model ParallelismM Shoeybi, M Patwary, R Puri, P LeGresley, J Casper, B CatanzaroarXiv preprint arXiv:1909.08053, 201912602019
2-) Scalable Bayesian Optimization Using Deep Neural NetworksJ Snoek, O Rippel, K Swersky, R Kiros, N Satish, N Sundaram, M Patwary, ...arXiv preprint arXiv:1502.05700, 201512222015
3-) Deep learning scaling is predictable, empiricallyJ Hestness, S Narang, N Ardalani, G Diamos, H Jun, H Kianinejad, ...arXiv preprint arXiv:1712.00409, 20176572017
4-) Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language ModelS Smith, M Patwary, B Norick, P LeGresley, S Rajbhandari, J Casper, ...arXiv preprint arXiv:2201.11990, 2022738*2022
5-) Efficient large-scale language model training on GPU clusters using megatron-LMD Narayanan, M Shoeybi, J Casper, P LeGresley, M Patwary, ...Proceedings of the International Conference for High Performance Computing …, 20215932021