The current COVID-19 outbreak has had a profound influence on public health and daily life. Despite all restrictions and vaccination programs, COVID-19 still can lead to fatality due to a lack of COVID-19-specific treatments. A number of studies have demonstrated the feasibility to develop therapeutics by targeting underlying components of the viral proteome. Here we reviewed recently developed and validated small molecule inhibitors of SARS-CoV-2's nonstructural proteins. We described the validation level of identified compounds specific for SARS-CoV-2 in the presence of in vitro and in vivo supporting data. The mechanisms of pharmacological activity, as well as approaches for developing improved SARS-CoV-2 NSP inhibitors have been emphasized.