Theorem 1 Let be a convex and a -smooth function. Then, for any ,

*Proof:* Let . Then,

where follows by using gradient inequality for the term since is a convex function, and using the -smoothess of for the term, and follows by substituting . 557 more words