BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Tyrus Berry

Tyrus Berry appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
math.DSmath.CAmath.NAmath.STMethodologyphysics.data-anStatistics TheoryMachine Learningmath.DGmath.FAmath.SPnlin.CDNumerical Analysisphysics.comp-ph

Catalog footprint

What is connected

17works
14topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

17 published item(s)

preprint2025arXiv

Dependence of Microstructure Classification Accuracy on Crystallographic Data Representation

Convolutional neural networks are increasingly being used to analyze and classify material microstructures, motivated by the possibility that they will be able to identify relevant microstructural features more efficiently and impartially than human experts. While up to now convolutional neural networks have mostly been applied to light optimal microscopy and scanning electron microscope micrographs, application to EBSD micrographs will be increasingly common as rational design generates materials with unknown textures and phase compositions. This raises the question of how crystallographic orientation should be represented in such a convolutional neural network, and whether this choice has a significant effect on the network's analysis and classification accuracy. Four representations of orientation information are examined and are used with convolutional neural networks to classify five synthetic microstructures with varying textures and grain geometries. Of these, a spectral embedding of crystallographic orientations in a space that respects the crystallographic symmetries performs by far the best, even when the network is trained on small volumes of data such as could be accessible by practical experiments.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

GiDR-DUN; Gradient Dimensionality Reduction -- Differences and Unification

TSNE and UMAP are two of the most popular dimensionality reduction algorithms due to their speed and interpretable low-dimensional embeddings. However, while attempts have been made to improve on TSNE's computational complexity, no existing method can obtain TSNE embeddings at the speed of UMAP. In this work, we show that this is indeed possible by combining the two approaches into a single method. We theoretically and experimentally evaluate the full space of parameters in the TSNE and UMAP algorithms and observe that a single parameter, the normalization, is responsible for switching between them. This, in turn, implies that a majority of the algorithmic differences can be toggled without affecting the embeddings. We discuss the implications this has on several theoretic claims underpinning the UMAP framework, as well as how to reconcile them with existing TSNE interpretations. Based on our analysis, we propose a new dimensionality reduction algorithm, GDR, that combines previously incompatible techniques from TSNE and UMAP and can replicate the results of either algorithm by changing the normalization. As a further advantage, GDR performs the optimization faster than available UMAP methods and thus an order of magnitude faster than available TSNE methods. Our implementation is plug-and-play with the traditional UMAP and TSNE libraries and can be found at github.com/Andrew-Draganov/GiDR-DUN.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Learning theory for dynamical systems

The task of modelling and forecasting a dynamical system is one of the oldest problems, and it remains challenging. Broadly, this task has two subtasks - extracting the full dynamical information from a partial observation; and then explicitly learning the dynamics from this information. We present a mathematical framework in which the dynamical information is represented in the form of an embedding. The framework combines the two subtasks using the language of spaces, maps, and commutations. The framework also unifies two of the most common learning paradigms - delay-coordinates and reservoir computing. We use this framework as a platform for two other investigations of the reconstructed system - its dynamical stability; and the growth of error under iterations. We show that these questions are deeply tied to more fundamental properties of the underlying system - the behavior of matrix cocycles over the base dynamics, its non-uniform hyperbolic behavior, and its decay of correlations. Thus, our framework bridges the gap between universally observed behavior of dynamics modelling; and the spectral, differential and ergodic properties intrinsic to the dynamics.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

A Higher Order Unscented Transform

We develop a new approach for estimating the expected values of nonlinear functions applied to multivariate random variables with arbitrary distributions. Rather than assuming a particular distribution, we assume that we are only given the first four moments of the distribution. The goal is to summarize the distribution using a small number of quadrature nodes which are called $σ$-points. We achieve this by choosing nodes and weights in order to match the specified moments of the distribution. The classical scaled unscented transform (SUT) matches the mean and covariance of a distribution. In this paper, introduce the higher order unscented transform (HOUT) which also matches any given skewness and kurtosis tensors. It turns out that the key to matching the higher moments is the rank-1 tensor decomposition. While the minimal rank-1 decomposition is NP-complete, we present a practical algorithm for computing a non-minimal rank-1 decomposition and prove convergence in linear time. We then show how to combine the rank-1 decompositions of the moments in order to form the $σ$-points and weights of the HOUT. By passing the $σ$-points through a nonlinear function and applying our quadrature rule we can estimate the moments of the output distribution. We prove that the HOUT is exact on arbitrary polynomials up to fourth order. Finally, we numerically compare the HOUT to the SUT on nonlinear functions applied to non-Gaussian random variables including an application to forecasting and uncertainty quantification for chaotic dynamics.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Bridging data science and dynamical systems theory

This short review describes mathematical techniques for statistical analysis and prediction in dynamical systems. Two problems are discussed, namely (i) the supervised learning problem of forecasting the time evolution of an observable under potentially incomplete observations at forecast initialization; and (ii) the unsupervised learning problem of identification of observables of the system with a coherent dynamical evolution. We discuss how ideas from from operator-theoretic ergodic theory combined with statistical learning theory provide an effective route to address these problems, leading to methods well-adapted to handle nonlinear dynamics, with convergence guarantees as the amount of training data increases.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Fractional Diffusion Maps

In this paper, we extend the diffusion maps algorithm on a family of heat kernels that are either local (having exponential decay) or nonlocal (having polynomial decay), arising in various applications. For example, these kernels have been used as a regularizer in various supervised learning tasks for denoising images. Importantly, these heat kernels give rise to operators that include (but are not restricted to) the generators of the classical Laplacian associated to Brownian processes as well as the fractional Laplacian associated with $β$-stable Lévy processes. For local kernels, while the method is a version of the diffusion maps algorithm, we show that the applications with non-Gaussian local heat kernels approximate temporally rescaled Laplace-Beltrami operators. For the non-local heat kernels, we modify the diffusion maps algorithm to estimate fractional Laplacian operators. Here, the graph distance is used to approximate the geodesic distance with appropriate error bounds. While this approximation becomes numerically expensive as the number of data points increases, it produces an accurate operator estimation that is robust to the choice of the kernel bandwidth parameter value. In contrast, the local kernels are numerically more efficient but more sensitive to the choice of kernel bandwidth parameter value. In an application to estimate non-smooth regression functions, we find that using the nonlocal kernel as a regularizer produces a more robust and accurate estimate than using local kernels. For manifolds with boundary, we find that the proposed fractional diffusion maps framework implemented with non-local kernels approximates the regional fractional Laplacian.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Spectral exterior calculus

A spectral approach to building the exterior calculus in manifold learning problems is developed. The spectral approach is shown to converge to the true exterior calculus in the limit of large data. Simultaneously, the spectral approach decouples the memory requirements from the amount of data points and ambient space dimension. To achieve this, the exterior calculus is reformulated entirely in terms of the eigenvalues and eigenfunctions of the Laplacian operator on functions. The exterior derivatives of these eigenfunctions (and their wedge products) are shown to form a frame (a type of spanning set) for appropriate $L^2$ spaces of $k$-forms, as well as higher-order Sobolev spaces. Formulas are derived to express the Laplace-de Rham operators on forms in terms of the eigenfunctions and eigenvalues of the Laplacian on functions. By representing the Laplace-de Rham operators in this frame, spectral convergence results are obtained via Galerkin approximation techniques. Numerical examples demonstrate accurate recovery of eigenvalues and eigenforms of the Laplace-de Rham operator on 1-forms. The correct Betti numbers are obtained from the kernel of this operator approximated from data sampled on several orientable and non-orientable manifolds, and the eigenforms are visualized via their corresponding vector fields. These vector fields form a natural orthonormal basis for the space of square-integrable vector fields, and are ordered by a Dirichlet energy functional which measures oscillatory behavior. The spectral framework also shows promising results on a non-smooth example (the Lorenz 63 attractor), suggesting that a spectral formulation of exterior calculus may be feasible in spaces with no differentiable structure.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Correcting biased observation model error in data assimilation

While the formulation of most data assimilation schemes assumes an unbiased observation model error, in real applications, model error with nontrivial biases is unavoidable. A practical example is the error in the radiative transfer model (which is used to assimilate satellite measurements) in the presence of clouds. As a consequence, many (in fact 99\%) of the cloudy observed measurements are not being used although they may contain useful information. This paper presents a novel nonparametric Bayesian scheme which is able to learn the observation model error distribution and correct the bias in incoming observations. This scheme can be used in tandem with any data assimilation forecasting system. The proposed model error estimator uses nonparametric likelihood functions constructed with data-driven basis functions based on the theory of kernel embeddings of conditional distributions developed in the machine learning community. Numerically, we show positive results with two examples. The first example is designed to produce a bimodality in the observation model error (typical of "cloudy" observations) by introducing obstructions to the observations which occur randomly in space and time. The second example, which is physically more realistic, is to assimilate cloudy satellite brightness temperature-like quantities, generated from a stochastic cloud model for tropical convection and a simple radiative transfer model.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Density Estimation on Manifolds with Boundary

Density estimation is a crucial component of many machine learning methods, and manifold learning in particular, where geometry is to be constructed from data alone. A significant practical limitation of the current density estimation literature is that methods have not been developed for manifolds with boundary, except in simple cases of linear manifolds where the location of the boundary is assumed to be known. We overcome this limitation by developing a density estimation method for manifolds with boundary that does not require any prior knowledge of the location of the boundary. To accomplish this we introduce statistics that provably estimate the distance and direction of the boundary, which allows us to apply a cut-and-normalize boundary correction. By combining multiple cut-and-normalize estimators we introduce a consistent kernel density estimator that has uniform bias, at interior and boundary points, on manifolds with boundary.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Forecasting Turbulent Modes with Nonparametric Diffusion Models: Learning from noisy data

In this paper, we apply a recently developed nonparametric modeling approach, the "diffusion forecast", to predict the time-evolution of Fourier modes of turbulent dynamical systems. While the diffusion forecasting method assumes the availability of a noise-free training data set observing the full state space of the dynamics, in real applications we often have only partial observations which are corrupted by noise. To alleviate these practical issues, following the theory of embedology, the diffusion model is built using the delay-embedding coordinates of the data. We show that this delay embedding biases the geometry of the data in a way which extracts the most stable component of the dynamics and reduces the influence of independent additive observation noise. The resulting diffusion forecast model approximates the semigroup solutions of the generator of the underlying dynamics in the limit of large data and when the observation noise vanishes. As in any standard forecasting problem, the forecasting skill depends crucially on the accuracy of the initial conditions. We introduce a novel Bayesian method for filtering the discrete-time noisy observations which works with the diffusion forecast to determine the forecast initial densities. Numerically, we compare this nonparametric approach with standard stochastic parametric models on a wide-range of well-studied turbulent modes, including the Lorenz-96 model in weakly chaotic to fully turbulent regimes and the barotropic modes of a quasi-geostrophic model with baroclinic instabilities. We show that when the only available data is the low-dimensional set of noisy modes that are being modeled, the diffusion forecast is indeed competitive to the perfect model.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Iterated Diffusion Maps for Feature Identification

Recently, the theory of diffusion maps was extended to a large class of local kernels with exponential decay which were shown to represent various Riemannian geometries on a data set sampled from a manifold embedded in Euclidean space. Moreover, local kernels were used to represent a diffeomorphism, H, between a data set and a feature of interest using an anisotropic kernel function, defined by a covariance matrix based on the local derivatives, DH. In this paper, we generalize the theory of local kernels to represent degenerate mappings where the intrinsic dimension of the data set is higher than the intrinsic dimension of the feature space. First, we present a rigorous method with asymptotic error bounds for estimating DH from the training data set and feature values. We then derive scaling laws for the singular values of the local linear structure of the data, which allows the identification the tangent space and improved estimation of the intrinsic dimension of the manifold and the bandwidth parameter of the diffusion maps algorithm. Using these numerical tools, our approach to feature identification is to iterate the diffusion map with appropriately chosen local kernels that emphasize the features of interest. We interpret the iterated diffusion map (IDM) as a discrete approximation to an intrinsic geometric flow which smoothly changes the geometry of the data space to emphasize the feature of interest. When the data lies on a product manifold of the feature manifold with an irrelevant manifold, we show that the IDM converges to the quotient manifold which is isometric to the feature manifold, thereby eliminating the irrelevant dimensions. We will also demonstrate empirically that if we apply the IDM to features that are not a quotient of the data space, the algorithm identifies an intrinsically lower-dimensional set embedding of the data which better represents the features.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Local Kernels and the Geometric Structure of Data

We introduce a theory of local kernels, which generalize the kernels used in the standard diffusion maps construction of nonparametric modeling. We prove that evaluating a local kernel on a data set gives a discrete representation of the generator of a continuous Markov process, which converges in the limit of large data. We explicitly connect the drift and diffusion coefficients of the process to the moments of the kernel. Moreover, when the kernel is symmetric, the generator is the Laplace-Beltrami operator with respect to a geometry which is influenced by the embedding geometry and the properties of the kernel. In particular, this allows us to generate any Riemannian geometry by an appropriate choice of local kernel. In this way, we continue a program of Belkin, Niyogi, Coifman and others to reinterpret the current diverse collection of kernel-based data analysis methods and place them in a geometric framework. We show how to use this framework to design local kernels invariant to various features of data. These data-driven local kernels can be used to construct conformally invariant embeddings and reconstruct global diffeomorphisms.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Nonparametric forecasting of low-dimensional dynamical systems

This letter presents a non-parametric modeling approach for forecasting stochastic dynamical systems on low-dimensional manifolds. The key idea is to represent the discrete shift maps on a smooth basis which can be obtained by the diffusion maps algorithm. In the limit of large data, this approach converges to a Galerkin projection of the semigroup solution to the underlying dynamics on a basis adapted to the invariant measure. This approach allows one to quantify uncertainties (in fact, evolve the probability distribution) for non-trivial dynamical systems with equation-free modeling. We verify our approach on various examples, ranging from an inhomogeneous anisotropic stochastic differential equation on a torus, the chaotic Lorenz three-dimensional model, and the Niño-3.4 data set which is used as a proxy of the El-Niño Southern Oscillation.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Nonparametric Uncertainty Quantification for Stochastic Gradient Flows

This paper presents a nonparametric statistical modeling method for quantifying uncertainty in stochastic gradient systems with isotropic diffusion. The central idea is to apply the diffusion maps algorithm to a training data set to produce a stochastic matrix whose generator is a discrete approximation to the backward Kolmogorov operator of the underlying dynamics. The eigenvectors of this stochastic matrix, which we will refer to as the diffusion coordinates, are discrete approximations to the eigenfunctions of the Kolmogorov operator and form an orthonormal basis for functions defined on the data set. Using this basis, we consider the projection of three uncertainty quantification (UQ) problems (prediction, filtering, and response) into the diffusion coordinates. In these coordinates, the nonlinear prediction and response problems reduce to solving systems of infinite-dimensional linear ordinary differential equations. Similarly, the continuous-time nonlinear filtering problem reduces to solving a system of infinite-dimensional linear stochastic differential equations. Solving the UQ problems then reduces to solving the corresponding truncated linear systems in finitely many diffusion coordinates. By solving these systems we give a model-free algorithm for UQ on gradient flow systems with isotropic diffusion. We numerically verify these algorithms on a 1-dimensional linear gradient flow system where the analytic solutions of the UQ problems are known. We also apply the algorithm to a chaotically forced nonlinear gradient flow system which is known to be well approximated as a stochastically forced gradient flow.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Semiparametric forecasting and filtering: correcting low-dimensional model error in parametric models

Semiparametric forecasting and filtering are introduced as a method of addressing model errors arising from unresolved physical phenomena. While traditional parametric models are able to learn high-dimensional systems from small data sets, their rigid parametric structure makes them vulnerable to model error. On the other hand, nonparametric models have a very flexible structure, but they suffer from the curse-of-dimensionality and are not practical for high-dimensional systems. The semiparametric approach loosens the structure of a parametric model by fitting a data-driven nonparametric model for the parameters. Given a parametric dynamical model and a noisy data set of historical observations, an adaptive Kalman filter is used to extract a time-series of the parameter values. A nonparametric forecasting model for the parameters is built by projecting the discrete shift map onto a data-driven basis of smooth functions. Existing techniques for filtering and forecasting algorithms extend naturally to the semiparametric model which can effectively compensate for model error, with forecasting skill approaching that of the perfect model. Semiparametric forecasting and filtering are a generalization of statistical semiparametric models to time-dependent distributions evolving under dynamical systems.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Variable Bandwidth Diffusion Kernels

Practical applications of kernel methods often use variable bandwidth kernels, also known as self-tuning kernels, however much of the current theory of kernel based techniques is only applicable to fixed bandwidth kernels. In this paper, we derive the asymptotic expansion of these variable bandwidth kernels for arbitrary bandwidth functions; generalizing the theory of Diffusion Maps and Laplacian Eigenmaps. We also derive pointwise error estimates for the corresponding discrete operators which are based on finite data sets; generalizing a result of Singer which was restricted to fixed bandwidth kernels. Our analysis reveals how areas of small sampling density lead to large errors, particularly for fixed bandwidth kernels. We explain the limitation of the existing theory to data sampled from compact manifolds by showing that when the sampling density is not bounded away from zero (which implies that the data lies on an open set) the error estimates for fixed bandwidth kernels will be unbounded. We show that this limitation can be overcome by choosing a bandwidth function inversely proportional to the sampling density (which can be estimated from data) which allows us to control the error estimates uniformly over a non-compact manifold. We numerically verify these results on non-compact manifolds by constructing the generator of the Ornstein-Uhlenbeck process on a real line and a two-dimensional plane using data sampled independently from the respective invariant measures. We also verify our results on compact manifolds by constructing the Laplacian on the unit circle and the unit sphere and we show that the variable bandwidth kernels exhibit reduced sensitivity to bandwidth selection and give better results for an automatic bandwidth selection algorithm.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Linear theory for filtering nonlinear multiscale systems with model error

We study filtering of multiscale dynamical systems with model error arising from unresolved smaller scale processes. The analysis assumes continuous-time noisy observations of all components of the slow variables alone. For a linear model with Gaussian noise, we prove existence of a unique choice of parameters in a linear reduced model for the slow variables. The linear theory extends to to a non-Gaussian, nonlinear test problem, where we assume we know the optimal stochastic parameterization and the correct observation model. We show that when the parameterization is inappropriate, parameters chosen for good filter performance may give poor equilibrium statistical estimates and vice versa. Given the correct parameterization, it is imperative to estimate the parameters simultaneously and to account for the nonlinear feedback of the stochastic parameters into the reduced filter estimates. In numerical experiments on the two-layer Lorenz-96 model, we find that parameters estimated online, as part of a filtering procedure, produce accurate filtering and equilibrium statistical prediction. In contrast, a linear regression based offline method, which fits the parameters to a given training data set independently from the filter, yields filter estimates which are worse than the observations or even divergent when the slow variables are not fully observed.

0 citations0 reviewsNo code linkedOpen access