Han's Memo
https://levelfour.github.io/blog/
Recent content on Han's Memo
Hugo  gohugo.io
en
© Han Bao 2018
Wed, 21 Mar 2018 10:50:23 +0900

Empirical Process in MEstimation (Chapter 3)
https://levelfour.github.io/blog/posts/empiricalprocesschapter3/
Wed, 21 Mar 2018 10:50:23 +0900
https://levelfour.github.io/blog/posts/empiricalprocesschapter3/
In Chapter 3, one of the most important theorem in empirical process theory, namely, the Uniform Law of Large Numbers (ULLN), is stated and discussed.
Summary If a function class $\mathcal{G}$ is said to satisfy the ULLN, then $$ \sup_{g \in \mathcal{G}} \left \int gd(P_n  P) \right \rightarrow 0, \quad \text{a.s.} $$ The ULLN means that the expected and empirical average become closer as the number of samples gets larger for any $g \in \mathcal{G}$, which is explained in the following figure.

Empirical Process in MEstimation (Chapter 2)
https://levelfour.github.io/blog/posts/empiricalprocesschapter2/
Sun, 11 Mar 2018 21:42:14 +0800
https://levelfour.github.io/blog/posts/empiricalprocesschapter2/
I would like to summarize the contents of van de Geer (2000)1 one chapter by one, which is devoted for the theory of the empirical process and the convergence rates.
In Chapter 2, the definitions of the empirical process and entropy are given.
Empirical Measure As the number of samples gets larger, the estimation error will converge to zero in many cases. This convergence property is stated by the empirical process.

Optimal Convergence Rate for Empirical Risk Minimization
https://levelfour.github.io/blog/posts/ermoptimalconvergencerate/
Fri, 23 Feb 2018 23:15:55 +0900
https://levelfour.github.io/blog/posts/ermoptimalconvergencerate/
These days, papers such as du Plessis et al. (NIPS2014)1 and Sakai et al. (ICML2017)2 refers optimality of convergence rates of their learning methods. The convergence rate $O(n^{\frac{1}{2}})$ is referred to as optimal (parametric) convergence rate for classification. In this article, I investigate why this is optimal for empirical risk minimization.
Most of the parts in this article try to describe intuition of Mendelson’s technical report3. I tried to explain as intuitive as possible, but sometimes sacrificed rigidness of discussion.