Source author record

Alexander P. Kreuzer

Explore this author’s publications, collaborators and research topics. This catalog record has not been claimed by its author.

ResearcherUnclaimed source record

math.LO Computation and Language Logic in Computer Science Machine Learning

Catalog footprint

Research overview

11works

4topics

4close collaborators

Actions

Is this your work?

Open graph Browse papers

preprint2020arXiv

WaLDORf: Wasteless Language-model Distillation On Reading-comprehension

Transformer based Very Large Language Models (VLLMs) like BERT, XLNet and RoBERTa, have recently shown tremendous performance on a large variety of Natural Language Understanding (NLU) tasks. However, due to their size, these VLLMs are extremely resource intensive and cumbersome to deploy at production time. Several recent publications have looked into various ways to distil knowledge from a transformer based VLLM (most commonly BERT-Base) into a smaller model which can run much faster at inference time. Here, we propose a novel set of techniques which together produce a task-specific hybrid convolutional and transformer model, WaLDORf, that achieves state-of-the-art inference speed while still being more accurate than previous distilled models.

preprint2015arXiv

A lower bound on Gowers' FIN_k theorem

Gowers' FIN $_k$ theorem, also called Gowers' pigeonhole principle or Gowers' theorem, is a Ramsey-type theorem. It first occurred in the study of Banach space theory and is a natural generalization of Hindman's theorem. In this short note, we will show that Gowers' FIN $_k$ theorem does not follow from ACA $_0$ .

preprint2015arXiv

Measure theory and higher order arithmetic

We investigate the statement that the Lebesgue measure defined on all subsets of the Cantor space exists. As base system we take $\mathsf{ACA}_0^ω+ (μ)$ . The system $\mathsf{ACA}_0^ω$ is the higher order extension of Friedman's system $\mathsf{ACA}_0$ , and $(μ)$ denotes Feferman's $μ$ , that is a uniform functional for arithmetical comprehension defined by $f(μ(f))=0$ if $\exists n f(n)=0$ for $f\in \mathbb{N}^\mathbb{N}$ . Feferman's $μ$ will provide countable unions and intersections of sets of reals and is, in fact, equivalent to this. For this reasons $\mathsf{ACA}_0^ω+ (μ)$ is the weakest fragment of higher order arithmetic where $σ$ -additive measures are directly definable. We obtain that over $\mathsf{ACA}_0^ω+ (μ)$ the existence of the Lebesgue measure is $Π^1_2$ -conservative over $\mathsf{ACA}_0^ω$ and with this conservative over $\mathsf{PA}$ . Moreover, we establish a corresponding program extraction result.

preprint2015arXiv

Minimal idempotent ultrafilters and the Auslander-Ellis theorem

We characterize the existence of minimal idempotent ultrafilters (on N) in the style of reverse mathematics and higher-order reverse mathematics using the Auslander-Ellis theorem and variant thereof. We obtain that the existence of minimal idempotent ultrafilters restricted to countable algebras of sets is equivalent to the Auslander-Ellis theorem (AET) and that the existence of minimal idempotent ultrafilters as higher-order objects is $Π^1_2$ -conservative over a refinement of AET.

preprint2015arXiv

On principles between $Σ_1$ - and $Σ_2$ -induction, and monotone enumerations

We show that many principles of first-order arithmetic, previously only known to lie strictly between $Σ_1$ -induction and $Σ_2$ -induction, are equivalent to the well-foundedness of $ω^ω$ . Among these principles are the iteration of partial functions ( $PΣ_1$ ) of Hájek and Paris, the bounded monotone enumerations principle (non-iterated, BME $_1$ ) by Chong, Slaman, and Yang, the relativized Paris-Harrington principle for pairs, and the totality of the relativized Ackermann-Péter function. With this we show that the well-foundedness of $ω^ω$ is a far more widespread than usually suspected. Further, we investigate the $k$ -iterated version of the bounded monotone iterations principle (BME $_k$ ), and show that it is equivalent to the well-foundedness of the $k+1$ -height $ω$ -tower.

preprint2014arXiv

Bounded variation and the strength of Helly's selection theorem

We analyze the strength of Helly's selection theorem HST, which is the most important compactness theorem on the space of functions of bounded variation. For this we utilize a new representation of this space intermediate between $L_1$ and the Sobolev space W1,1, compatible with the, so called, weak* topology. We obtain that HST is instance-wise equivalent to the Bolzano-Weierstraß principle over RCA0. With this HST is equivalent to ACA0 over RCA0. A similar classification is obtained in the Weihrauch lattice.

preprint2013arXiv

On idempotent ultrafilters in higher-order reverse mathematics

We analyze the strength of the existence of idempotent ultrafilters in higher-order reverse mathematics. Let (Uidem) be the statement that an idempotent ultrafilter on the natural numbers exists. We show that over ACA_0^w, the higher-order extension of ACA_0, the statement (Uidem) implies the iterated Hindman's theorem (IHT), and we show that ACA_0^w + (Uidem) is Pi^1_2-conservative over ACA_0^w + IHT and thus over ACA_0^+.

preprint2012arXiv

From Bolzano-Weierstraß to Arzelà-Ascoli

We show how one can obtain solutions to the Arzelà-Ascoli theorem using suitable applications of the Bolzano-Weierstraß principle. With this, we can apply the results from \cite{aK} and obtain a classification of the strength of instances of the Arzelà-Ascoli theorem and a variant of it. Let AA be the statement that each equicontinuous sequence of functions f_n: [0,1] --> [0,1] contains a subsequence that converges uniformly with the rate 2^-k and let AA_weak be the statement that each such sequence contains a subsequence which converges uniformly but possibly without any rate. We show that AA is instance-wise equivalent over RCA_0 to the Bolzano-Weierstraß principle BW and that AA_weak is instance-wise equivalent over WKL_0 to BW_weak, and thus to the strong cohesive principle StCOH. Moreover, we show that over RCA_0 the principles AA_weak, BW_weak + WKL and StCOH + WKL are equivalent.

preprint2011arXiv

Non-principal ultrafilters, program extraction and higher order reverse mathematics

We investigate the strength of the existence of a non-principal ultrafilter over fragments of higher order arithmetic. Let U be the statement that a non-principal ultrafilter exists and let ACA_0^ω be the higher order extension of ACA_0. We show that ACA_0^ω+U is Π^1_2-conservative over ACA_0^ω and thus that ACA_0^ω+\U is conservative over PA. Moreover, we provide a program extraction method and show that from a proof of a strictly Π^1_2 statement \forall f \exists g A(f,g) in ACA_0^ω+U a realizing term in Gödel's system T can be extracted. This means that one can extract a term t, such that A(f,t(f)).

preprint2011arXiv

On the strength of weak compactness

We study the logical and computational strength of weak compactness in the separable Hilbert space \ell_2. Let weak-BW be the statement the every bounded sequence in \ell_2 has a weak cluster point. It is known that weak-BW is equivalent to ACA_0 over RCA_0 and thus that it is equivalent to (nested uses of) the usual Bolzano-Weierstraß principle BW. We show that weak-BW is instance-wise equivalent to the Π^0_2-CA. This means that for each Π^0_2 sentence A(n) there is a sequence (x_i) in \ell_2, such that one can define the comprehension functions for A(n) recursively in a cluster point of (x_i). As consequence we obtain that the Turing degrees d > 0" are exactly those degrees that contain a weak cluster point of any computable, bounded sequence in \ell_2. Since a cluster point of any sequence in the unit interval [0,1] can be computed in a degree low over 0', this show also that instances of weak-BW are strictly stronger than instances of BW. We also comment on the strength of weak-BW in the context of abstract Hilbert spaces in the sense of Kohlenbach and show that his construction of a solution for the functional interpretation of weak compactness is optimal.

preprint2011arXiv

The cohesive principle and the Bolzano-Weierstraß principle

The aim of this paper is to determine the logical and computational strength of instances of the Bolzano-Weierstraß principle (BW) and a weak variant of it. We show that BW is instance-wise equivalent to the weak König's lemma for $Σ^0_1$ -trees ( $Σ^0_1$ -WKL). This means that from every bounded sequence of reals one can compute an infinite $Σ^0_1$ -0/1-tree, such that each infinite branch of it yields an accumulation point and vice versa. Especially, this shows that the degrees d >> 0' are exactly those containing an accumulation point for all bounded computable sequences. Let BW_weak be the principle stating that every bounded sequence of real numbers contains a Cauchy subsequence (a sequence converging but not necessarily fast). We show that BW_weak is instance-wise equivalent to the (strong) cohesive principle (StCOH) and - using this - obtain a classification of the computational and logical strength of BW_weak. Especially we show that BW_weak does not solve the halting problem and does not lead to more than primitive recursive growth. Therefore it is strictly weaker than BW. We also discuss possible uses of BW_weak.

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Alexander P. Kreuzer

Research overview

Is this your work?

11 publications

WaLDORf: Wasteless Language-model Distillation On Reading-comprehension

A lower bound on Gowers' FIN_k theorem

Measure theory and higher order arithmetic

Minimal idempotent ultrafilters and the Auslander-Ellis theorem

On principles between $Σ_1$ - and $Σ_2$ -induction, and monotone enumerations

Bounded variation and the strength of Helly's selection theorem

On idempotent ultrafilters in higher-order reverse mathematics

From Bolzano-Weierstraß to Arzelà-Ascoli

Non-principal ultrafilters, program extraction and higher order reverse mathematics

On the strength of weak compactness

The cohesive principle and the Bolzano-Weierstraß principle

See the researcher in context

Building this map preview

Alexander P. Kreuzer

Research overview

Is this your work?

11 publications

WaLDORf: Wasteless Language-model Distillation On Reading-comprehension

A lower bound on Gowers' FIN_k theorem

Measure theory and higher order arithmetic

Minimal idempotent ultrafilters and the Auslander-Ellis theorem

On principles between Σ1Σ_1Σ1​- and Σ2Σ_2Σ2​-induction, and monotone enumerations

Bounded variation and the strength of Helly's selection theorem

On idempotent ultrafilters in higher-order reverse mathematics

From Bolzano-Weierstraß to Arzelà-Ascoli

Non-principal ultrafilters, program extraction and higher order reverse mathematics

On the strength of weak compactness

The cohesive principle and the Bolzano-Weierstraß principle

See the researcher in context

Building this map preview

On principles between $Σ_1$ - and $Σ_2$ -induction, and monotone enumerations