Source author record

Kai Fong Ernest Chong

Kai Fong Ernest Chong appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

math.CO Machine Learning math.AC Artificial Intelligence Computer Vision Information Theory math.AG math.FA math.IT

Catalog footprint

What is connected

6works

9topics

4close collaborators

Actions

Connect this record

Open graph Browse works

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2022arXiv

FedCorr: Multi-Stage Federated Learning for Label Noise Correction

Federated learning (FL) is a privacy-preserving distributed learning paradigm that enables clients to jointly train a global model. In real-world FL implementations, client data could have label noise, and different clients could have vastly different label noise levels. Although there exist methods in centralized learning for tackling label noise, such methods do not perform well on heterogeneous label noise in FL settings, due to the typically smaller sizes of client datasets and data privacy requirements in FL. In this paper, we propose $\texttt{FedCorr}$, a general multi-stage framework to tackle heterogeneous label noise in FL, without making any assumptions on the noise models of local clients, while still maintaining client data privacy. In particular, (1) $\texttt{FedCorr}$ dynamically identifies noisy clients by exploiting the dimensionalities of the model prediction subspaces independently measured on all clients, and then identifies incorrect labels on noisy clients based on per-sample losses. To deal with data heterogeneity and to increase training stability, we propose an adaptive local proximal regularization term that is based on estimated local noise levels. (2) We further finetune the global model on identified clean clients and correct the noisy labels for the remaining noisy clients after finetuning. (3) Finally, we apply the usual training on all clients to make full use of all local data. Experiments conducted on CIFAR-10/100 with federated synthetic label noise, and on a real-world noisy dataset, Clothing1M, demonstrate that $\texttt{FedCorr}$ is robust to label noise and substantially outperforms the state-of-the-art methods at multiple noise levels.

preprint2020arXiv

A closer look at the approximation capabilities of neural networks

The universal approximation theorem, in one of its most general versions, says that if we consider only continuous activation functions $σ$, then a standard feedforward neural network with one hidden layer is able to approximate any continuous multivariate function $f$ to any given approximation threshold $\varepsilon$, if and only if $σ$ is non-polynomial. In this paper, we give a direct algebraic proof of the theorem. Furthermore we shall explicitly quantify the number of hidden units required for approximation. Specifically, if $X\subseteq \mathbb{R}^n$ is compact, then a neural network with $n$ input units, $m$ output units, and a single hidden layer with $\binom{n+d}{d}$ hidden units (independent of $m$ and $\varepsilon$), can uniformly approximate any polynomial function $f:X \to \mathbb{R}^m$ whose total degree is at most $d$ for each of its $m$ coordinate functions. In the general case that $f$ is any continuous function, we show there exists some $N\in \mathcal{O}(\varepsilon^{-n})$ (independent of $m$), such that $N$ hidden units would suffice to approximate $f$. We also show that this uniform approximation property (UAP) still holds even under seemingly strong conditions imposed on the weights. We highlight several consequences: (i) For any $δ> 0$, the UAP still holds if we restrict all non-bias weights $w$ in the last layer to satisfy $|w| < δ$. (ii) There exists some $λ>0$ (depending only on $f$ and $σ$), such that the UAP still holds if we restrict all non-bias weights $w$ in the first layer to satisfy $|w|>λ$. (iii) If the non-bias weights in the first layer are \emph{fixed} and randomly chosen from a suitable range, then the UAP holds with probability $1$.

preprint2014arXiv

Generalized Macaulay representations and the flag $f$-vectors of generalized colored complexes

A colored complex of type $\mathbf{a} = (a_1, \dots, a_n)$ is a simplicial complex $Δ$ on a vertex set $V$, together with an ordered partition $(V_1, \dots, V_n)$ of $V$, such that every face $F$ of $Δ$ satisfies $|F \cap V_i| \leq a_i$. For each $\mathbf{b} = (b_1, \dots, b_n) \leq \mathbf{a}$, let $f_{\mathbf{b}}$ be the number of faces $F$ of $Δ$ such that $|F \cap V_i| = b_i$. The array of integers $\{f_{\mathbf{b}}\}_{\mathbf{b} \leq \mathbf{a}}$ is called the fine $f$-vector of $Δ$, and it is a refinement of the $f$-vector of $Δ$. In this paper, we generalize the notion of Macaulay representations and give a numerical characterization of the fine $f$-vectors of colored complexes of arbitrary type, in terms of these generalized Macaulay representations. As part of the proof, we introduce the property of $\mathbf{a}$-Macaulay decomposability for simplicial complexes, which implies vertex-decomposability, and we show that every pure color-shifted balanced complex $Δ$ of type $\mathbf{a}$ is $\mathbf{a}$-Macaulay decomposable. Combined with previously known results, we also obtain a numerical characterization of the flag $f$-vectors of completely balanced Cohen-Macaulay complexes.

preprint2014arXiv

Hilbert functions of colored quotient rings and a generalization of the Clements-Lindström theorem

Given a polynomial ring $S = \Bbbk[x_1, \dots, x_n]$ over a field $\Bbbk$, and a monomial ideal $M$ of $S$, we say the quotient ring $R = S/M$ is Macaulay-Lex if for every graded ideal of $R$, there exists a lexicographic ideal of $R$ with the same Hilbert function. In this paper, we introduce a class of quotient rings with combinatorial significance, which we call colored quotient rings. This class of rings include Clements-Lindström rings and colored squarefree rings as special cases that are known to be Macaulay-Lex. We construct two new classes of Macaulay-Lex rings, characterize all colored quotient rings that are Macaulay-Lex, and give a simultaneous generalization of both the Clements-Lindström theorem and the Frankl-Füredi-Kalai theorem. We also show that the $f$-vectors of $(a_1, \dots, a_n)$-colored simplicial complexes or multicomplexes are never characterized by "reverse-lexicographic" complexes or multicomplexes when $n>1$ and $(a_1, \dots, a_n) \neq (1, \dots, 1)$.

preprint2013arXiv

An application of liaison theory to the Eisenbud-Green-Harris conjecture

In this paper, we apply liaison theory to the Eisenbud-Green-Harris conjecture and prove that the conjecture holds for a certain subclass of homogeneous ideals in the linkage class of a complete intersection ideal. In the case of three variables, we prove that the conjecture holds for Gorenstein ideals.

preprint2010arXiv

Fountain Codes with Varying Probability Distributions

Fountain codes are rateless erasure-correcting codes, i.e., an essentially infinite stream of encoded packets can be generated from a finite set of data packets. Several fountain codes have been proposed recently to minimize overhead, many of which involve modifications of the Luby transform (LT) code. These fountain codes, like the LT code, have the implicit assumption that the probability distribution is fixed throughout the encoding process. In this paper, we will use the theory of posets to show that this assumption is unnecessary, and by dropping it, we can achieve overhead reduction by as much as 64% lower than LT codes. We also present the fundamental theory of probability distribution designs for fountain codes with non-constant probability distributions that minimize overhead.