Catalog footprint

What is connected

48works
26topics
4close collaborators

Actions

Connect this record

Log in to claim

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

48 published item(s)

preprint2026arXiv

Chebyshev-Augmented One-Shot Transfer Learning for PINNs on Nonlinear Differential Equations

Physics-Informed Neural Networks (PINNs) offer a flexible paradigm for solving differential equations by embedding governing laws into the training objective. A persistent limitation is instance specificity: standard PINNs typically require retraining for each new forcing term, boundary/initial condition, or parameter setting. One-shot transfer learning (OTL) addresses this bottleneck for linear operators by freezing a pretrained latent representation and computing optimal output weights in closed form, but for nonlinear problems closed-form adaptation is generally unavailable because the loss is nonconvex in the output layer. In this paper we substantially broaden the class of nonlinearities amenable to one-shot PINN transfer by combining OTL with Chebyshev polynomial surrogates. We approximate general smooth weakly nonlinear terms by truncated Chebyshev expansions over a prescribed solution range, yielding a polynomial nonlinearity that can be handled by a perturbative decomposition into linear subproblems. A multi-head PINN learns a reusable latent space associated with the dominant linear operator; at test time, solutions to new instances are obtained via a sequence of closed-form linear solves in the output layer, without retraining the network body. We provide a unified derivation of the framework for ODEs and PDEs and demonstrate accuracy and fast online adaptation on nonlinear benchmarks, including non-polynomial and singular ODE nonlinearities as well as a reaction-diffusion PDE with saturating kinetics, demonstrating the method's utility in many-query regimes.

preprint2026arXiv

Gradient Scaling Effects in Adaptive Spectral PINNs for Stiff Nonlinear ODEs

Physics-Informed Neural Networks (PINNs) often struggle to train reliably on stiff and oscillatory dynamical systems due to poor optimization conditioning. While prior work has emphasized representational remedies such as spectral parameterizations, the optimization implications of initial-condition (IC) embeddings in adaptive spectral PINNs have not been well characterized. In this work, we show that the choice of IC gating function induces explicit time-dependent gradient scaling, which interacts with spectral representations during training. Using a nonlinear stiff spring-pendulum ODE as a controlled benchmark, we compare exponential and linear IC gates in combination with fixed and adaptive Fourier spectral trunks. We observe stiffness-dependent changes in relative dominance for adaptive PINNs: at moderate stiffness ($k=20$), exponential gating often yields lower error but exhibits heterogeneous behavior across random seeds, whereas at higher stiffness ($k=60$), linear gating becomes preferable, with additional reversals observed at larger $k$. These trends hold for both relative $L^2$ error and maximum pointwise error and are confirmed by paired Wilcoxon signed-rank tests with Holm correction. Overall, our results demonstrate that IC embeddings are not a neutral design choice in PINNs: the induced gradient scaling materially shapes optimization conditioning in stiff regimes, with distinct sensitivity patterns in baseline and adaptive spectral models.

preprint2026arXiv

Improving Router Security using BERT

Previous work on home router security has shown that using system calls to train a transformer-based language model built on a BERT-style encoder using contrastive learning is effective in detecting several types of malware, but the performance remains limited at low false positive rates. In this work, we demonstrate that using a high-fidelity eBPF-based system call sensor, together with contrastive augmented learning (which introduces controlled mutations of negative samples), improves detection performance at a low false positive rate. In addition, we introduce a network packet abstraction language that enables the creation of a pipeline similar to network packet data, and we show that network behavior provides complementary detection signals-yielding improved performance for network-focused malware at low false positive rates. Lastly, we implement these methods in an online router anomaly detection framework to validate the approach in an Internet of Things (IoT) deployment environment.

preprint2026arXiv

PTL-PINNs: Perturbation-Guided Transfer Learning with Physics- Informed Neural Networks for Nonlinear Systems

Accurately and efficiently solving nonlinear differential equations is crucial for modeling dynamic behavior across science and engineering. Physics-Informed Neural Networks (PINNs) have emerged as a powerful solution that embeds physical laws in training by enforcing equation residuals. However, these struggle to model nonlinear dynamics, suffering from limited generalization across problems and long training times. To address these limitations, we propose a perturbation-guided transfer learning framework for PINNs (PTL-PINN), which integrates perturbation theory with transfer learning to efficiently solve nonlinear equations. Unlike gradient-based transfer learning, PTL-PINNs solve an approximate linear perturbative system using closed-form expressions, enabling rapid generalization with the time complexity of matrix-vector multiplication. We show that PTL-PINNs achieve accuracy comparable to various Runge-Kutta methods, with computational speeds up to one order of magnitude faster. To benchmark performance, we solve a broad set of problems, including nonlinear oscillators across various damping regimes, the equilibrium-centered Lotka-Volterra system, the KPP-Fisher and the Wave equation. Since perturbation theory sets the accuracy bound of PTL-PINNs, we systematically evaluate its practical applicability. This work connects long-standing perturbation methods with PINNs, demonstrating how perturbation theory can guide foundational models to solve nonlinear systems with speeds comparable to those of classical solvers.

preprint2022arXiv

Encoding Involutory Invariances in Neural Networks

In certain situations, neural networks are trained upon data that obey underlying symmetries. However, the predictions do not respect the symmetries exactly unless embedded in the network structure. In this work, we introduce architectures that embed a special kind of symmetry namely, invariance with respect to involutory linear/affine transformations up to parity $p=\pm 1$. We provide rigorous theorems to show that the proposed network ensures such an invariance and present qualitative arguments for a special universal approximation theorem. An adaption of our techniques to CNN tasks for datasets with inherent horizontal/vertical reflection symmetry is demonstrated. Extensive experiments indicate that the proposed model outperforms baseline feed-forward and physics-informed neural networks while identically respecting the underlying symmetry.

preprint2022arXiv

Evaluating Error Bound for Physics-Informed Neural Networks on Linear Dynamical Systems

There have been extensive studies on solving differential equations using physics-informed neural networks. While this method has proven advantageous in many cases, a major criticism lies in its lack of analytical error bounds. Therefore, it is less credible than its traditional counterparts, such as the finite difference method. This paper shows that one can mathematically derive explicit error bounds for physics-informed neural networks trained on a class of linear systems of differential equations. More importantly, evaluating such error bounds only requires evaluating the differential equation residual infinity norm over the domain of interest. Our work shows a link between network residuals, which is known and used as loss function, and the absolute error of solution, which is generally unknown. Our approach is semi-phenomonological and independent of knowledge of the actual solution or the complexity or architecture of the network. Using the method of manufactured solution on linear ODEs and system of linear ODEs, we empirically verify the error evaluation algorithm and demonstrate that the actual error strictly lies within our derived bound.

preprint2022arXiv

Hamiltonian neural networks for solving equations of motion

There has been a wave of interest in applying machine learning to study dynamical systems. We present a Hamiltonian neural network that solves the differential equations that govern dynamical systems. This is an equation-driven machine learning method where the optimization process of the network depends solely on the predicted functions without using any ground truth data. The model learns solutions that satisfy, up to an arbitrarily small error, Hamilton's equations and, therefore, conserve the Hamiltonian invariants. The choice of an appropriate activation function drastically improves the predictability of the network. Moreover, an error analysis is derived and states that the numerical errors depend on the overall network performance. The Hamiltonian network is then employed to solve the equations for the nonlinear oscillator and the chaotic Henon-Heiles dynamical system. In both systems, a symplectic Euler integrator requires two orders more evaluation points than the Hamiltonian network in order to achieve the same order of the numerical error in the predicted phase space trajectories.

preprint2022arXiv

Improving Astronomical Time-series Classification via Data Augmentation with Generative Adversarial Networks

Due to the latest advances in technology, telescopes with significant sky coverage will produce millions of astronomical alerts per night that must be classified both rapidly and automatically. Currently, classification consists of supervised machine learning algorithms whose performance is limited by the number of existing annotations of astronomical objects and their highly imbalanced class distributions. In this work, we propose a data augmentation methodology based on Generative Adversarial Networks (GANs) to generate a variety of synthetic light curves from variable stars. Our novel contributions, consisting of a resampling technique and an evaluation metric, can assess the quality of generative models in unbalanced datasets and identify GAN-overfitting cases that the Fréchet Inception Distance does not reveal. We applied our proposed model to two datasets taken from the Catalina and Zwicky Transient Facility surveys. The classification accuracy of variable stars is improved significantly when training with synthetic data and testing with real data with respect to the case of using only real data.

preprint2022arXiv

Multi-Task Learning based Convolutional Models with Curriculum Learning for the Anisotropic Reynolds Stress Tensor in Turbulent Duct Flow

The Reynolds-averaged Navier-Stokes (RANS) equations require accurate modeling of the anisotropic Reynolds stress tensor. Traditional closure models, while sophisticated, often only apply to restricted flow configurations. Researchers have started using machine learning approaches to tackle this problem by developing more general closure models informed by data. In this work we build upon recent convolutional neural network architectures used for turbulence modeling and propose a multi-task learning-based fully convolutional neural network that is able to accurately predict the normalized anisotropic Reynolds stress tensor for turbulent duct flows. Furthermore, we also explore the application of curriculum learning to data-driven turbulence modeling.

preprint2022arXiv

One-Shot Transfer Learning of Physics-Informed Neural Networks

Solving differential equations efficiently and accurately sits at the heart of progress in many areas of scientific research, from classical dynamical systems to quantum mechanics. There is a surge of interest in using Physics-Informed Neural Networks (PINNs) to tackle such problems as they provide numerous benefits over traditional numerical approaches. Despite their potential benefits for solving differential equations, transfer learning has been under explored. In this study, we present a general framework for transfer learning PINNs that results in one-shot inference for linear systems of both ordinary and partial differential equations. This means that highly accurate solutions to many unknown differential equations can be obtained instantaneously without retraining an entire network. We demonstrate the efficacy of the proposed deep learning approach by solving several real-world problems, such as first- and second-order linear ordinary equations, the Poisson equation, and the time-dependent Schrodinger complex-value partial differential equation.

preprint2022arXiv

Physics-Informed Neural Networks for Quantum Eigenvalue Problems

Eigenvalue problems are critical to several fields of science and engineering. We expand on the method of using unsupervised neural networks for discovering eigenfunctions and eigenvalues for differential eigenvalue problems. The obtained solutions are given in an analytical and differentiable form that identically satisfies the desired boundary conditions. The network optimization is data-free and depends solely on the predictions of the neural network. We introduce two physics-informed loss functions. The first, called ortho-loss, motivates the network to discover pair-wise orthogonal eigenfunctions. The second loss term, called norm-loss, requests the discovery of normalized eigenfunctions and is used to avoid trivial solutions. We find that embedding even or odd symmetries to the neural network architecture further improves the convergence for relevant problems. Lastly, a patience condition can be used to automatically recognize eigenfunction solutions. This proposed unsupervised learning method is used to solve the finite well, multiple finite wells, and hydrogen atom eigenvalue quantum problems.

preprint2022arXiv

RcTorch: a PyTorch Reservoir Computing Package with Automated Hyper-Parameter Optimization

Reservoir computers (RCs) are among the fastest to train of all neural networks, especially when they are compared to other recurrent neural networks. RC has this advantage while still handling sequential data exceptionally well. However, RC adoption has lagged other neural network models because of the model's sensitivity to its hyper-parameters (HPs). A modern unified software package that automatically tunes these parameters is missing from the literature. Manually tuning these numbers is very difficult, and the cost of traditional grid search methods grows exponentially with the number of HPs considered, discouraging the use of the RC and limiting the complexity of the RC models which can be devised. We address these problems by introducing RcTorch, a PyTorch based RC neural network package with automated HP tuning. Herein, we demonstrate the utility of RcTorch by using it to predict the complex dynamics of a driven pendulum being acted upon by varying forces. This work includes coding examples. Example Python Jupyter notebooks can be found on our GitHub repository https://github.com/blindedjoy/RcTorch and documentation can be found at https://rctorch.readthedocs.io/.

preprint2021arXiv

A New Artificial Neuron Proposal with Trainable Simultaneous Local and Global Activation Function

The activation function plays a fundamental role in the artificial neural network learning process. However, there is no obvious choice or procedure to determine the best activation function, which depends on the problem. This study proposes a new artificial neuron, named global-local neuron, with a trainable activation function composed of two components, a global and a local. The global component term used here is relative to a mathematical function to describe a general feature present in all problem domain. The local component is a function that can represent a localized behavior, like a transient or a perturbation. This new neuron can define the importance of each activation function component in the learning phase. Depending on the problem, it results in a purely global, or purely local, or a mixed global and local activation function after the training phase. Here, the trigonometric sine function was employed for the global component and the hyperbolic tangent for the local component. The proposed neuron was tested for problems where the target was a purely global function, or purely local function, or a composition of two global and local functions. Two classes of test problems were investigated, regression problems and differential equations solving. The experimental tests demonstrated the Global-Local Neuron network's superior performance, compared with simple neural networks with sine or hyperbolic tangent activation function, and with a hybrid network that combines these two simple neural networks.

preprint2021arXiv

Convolutional Neural Network Models and Interpretability for the Anisotropic Reynolds Stress Tensor in Turbulent One-dimensional Flows

The Reynolds-averaged Navier-Stokes (RANS) equations are widely used in turbulence applications. They require accurately modeling the anisotropic Reynolds stress tensor, for which traditional Reynolds stress closure models only yield reliable results in some flow configurations. In the last few years, there has been a surge of work aiming at using data-driven approaches to tackle this problem. The majority of previous work has focused on the development of fully-connected networks for modeling the anisotropic Reynolds stress tensor. In this paper, we expand upon recent work for turbulent channel flow and develop new convolutional neural network (CNN) models that are able to accurately predict the normalized anisotropic Reynolds stress tensor. We apply the new CNN model to a number of one-dimensional turbulent flows. Additionally, we present interpretability techniques that help drive the model design and provide guidance on the model behavior in relation to the underlying physics.

preprint2021arXiv

Matching Embeddings for Domain Adaptation

In this work we address the problem of transferring knowledge obtained from a vast annotated source domain to a low labeled target domain. We propose Adversarial Variational Domain Adaptation (AVDA), a semi-supervised domain adaptation method based on deep variational embedded representations. We use approximate inference and domain adversarial methods to map samples from source and target domains into an aligned class-dependent embedding defined as a Gaussian Mixture Model. AVDA works as a classifier and considers a generative model that helps this classification. We used digits dataset for experimentation. Our results show that on a semi-supervised few-shot scenario our model outperforms previous methods in most of the adaptation tasks, even using a fewer number of labeled samples per class on target domain.

preprint2020arXiv

Application of Machine Learning to Predict the Risk of Alzheimer's Disease: An Accurate and Practical Solution for Early Diagnostics

Alzheimer's Disease (AD) ravages the cognitive ability of more than 5 million Americans and creates an enormous strain on the health care system. This paper proposes a machine learning predictive model for AD development without medical imaging and with fewer clinical visits and tests, in hopes of earlier and cheaper diagnoses. That earlier diagnoses could be critical in the effectiveness of any drug or medical treatment to cure this disease. Our model is trained and validated using demographic, biomarker and cognitive test data from two prominent research studies: Alzheimer's Disease Neuroimaging Initiative (ADNI) and Australian Imaging, Biomarker Lifestyle Flagship Study of Aging (AIBL). We systematically explore different machine learning models, pre-processing methods and feature selection techniques. The most performant model demonstrates greater than 90% accuracy and recall in predicting AD, and the results generalize across sub-studies of ADNI and to the independent AIBL study. We also demonstrate that these results are robust to reducing the number of clinical visits or tests per visit. Using a metaclassification algorithm and longitudinal data analysis we are able to produce a "lean" diagnostic protocol with only 3 tests and 4 clinical visits that can predict Alzheimer's development with 87% accuracy and 79% recall. This novel work can be adapted into a practical early diagnostic tool for predicting the development of Alzheimer's that maximizes accuracy while minimizing the number of necessary diagnostic tests and clinical visits.

preprint2020arXiv

Gravitational Wave Detection and Information Extraction via Neural Networks

Laser Interferometer Gravitational-Wave Observatory (LIGO) was the first laboratory to measure the gravitational waves. It was needed an exceptional experimental design to measure distance changes much less than a radius of a proton. In the same way, the data analyses to confirm and extract information is a tremendously hard task. Here, it is shown a computational procedure base on artificial neural networks to detect a gravitation wave event and extract the knowledge of its ring-down time from the LIGO data. With this proposal, it is possible to make a probabilistic thermometer for gravitational wave detection and obtain physical information about the astronomical body system that created the phenomenon. Here, the ring-down time is determined with a direct data measure, without the need to use numerical relativity techniques and high computational power.

preprint2020arXiv

MPCC: Matching Priors and Conditionals for Clustering

Clustering is a fundamental task in unsupervised learning that depends heavily on the data representation that is used. Deep generative models have appeared as a promising tool to learn informative low-dimensional data representations. We propose Matching Priors and Conditionals for Clustering (MPCC), a GAN-based model with an encoder to infer latent variables and cluster categories from data, and a flexible decoder to generate samples from a conditional latent space. With MPCC we demonstrate that a deep generative model can be competitive/superior against discriminative methods in clustering tasks surpassing the state of the art over a diverse set of benchmark datasets. Our experiments show that adding a learnable prior and augmenting the number of encoder updates improve the quality of the generated samples, obtaining an inception score of 9.49 $\pm$ 0.15 and improving the Fréchet inception distance over the state of the art by a 46.9% in CIFAR10.

preprint2020arXiv

Scalable End-to-end Recurrent Neural Network for Variable star classification

During the last decade, considerable effort has been made to perform automatic classification of variable stars using machine learning techniques. Traditionally, light curves are represented as a vector of descriptors or features used as input for many algorithms. Some features are computationally expensive, cannot be updated quickly and hence for large datasets such as the LSST cannot be applied. Previous work has been done to develop alternative unsupervised feature extraction algorithms for light curves, but the cost of doing so still remains high. In this work, we propose an end-to-end algorithm that automatically learns the representation of light curves that allows an accurate automatic classification. We study a series of deep learning architectures based on Recurrent Neural Networks and test them in automated classification scenarios. Our method uses minimal data preprocessing, can be updated with a low computational cost for new observations and light curves, and can scale up to massive datasets. We transform each light curve into an input matrix representation whose elements are the differences in time and magnitude, and the outputs are classification probabilities. We test our method in three surveys: OGLE-III, Gaia and WISE. We obtain accuracies of about $95\%$ in the main classes and $75\%$ in the majority of subclasses. We compare our results with the Random Forest classifier and obtain competitive accuracies while being faster and scalable. The analysis shows that the computational complexity of our approach grows up linearly with the light curve size, while the traditional approach cost grows as $N\log{(N)}$.

preprint2020arXiv

Solving Differential Equations Using Neural Network Solution Bundles

The time evolution of dynamical systems is frequently described by ordinary differential equations (ODEs), which must be solved for given initial conditions. Most standard approaches numerically integrate ODEs producing a single solution whose values are computed at discrete times. When many varied solutions with different initial conditions to the ODE are required, the computational cost can become significant. We propose that a neural network be used as a solution bundle, a collection of solutions to an ODE for various initial states and system parameters. The neural network solution bundle is trained with an unsupervised loss that does not require any prior knowledge of the sought solutions, and the resulting object is differentiable in initial conditions and system parameters. The solution bundle exhibits fast, parallelizable evaluation of the system state, facilitating the use of Bayesian inference for parameter estimation in real dynamical systems.

preprint2020arXiv

Unsupervised Learning of Solutions to Differential Equations with Generative Adversarial Networks

Solutions to differential equations are of significant scientific and engineering relevance. Recently, there has been a growing interest in solving differential equations with neural networks. This work develops a novel method for solving differential equations with unsupervised neural networks that applies Generative Adversarial Networks (GANs) to \emph{learn the loss function} for optimizing the neural network. We present empirical results showing that our method, which we call Differential Equation GAN (DEQGAN), can obtain multiple orders of magnitude lower mean squared errors than an alternative unsupervised neural network method based on (squared) $L_2$, $L_1$, and Huber loss functions. Moreover, we show that DEQGAN achieves solution accuracy that is competitive with traditional numerical methods. Finally, we analyze the stability of our approach and find it to be sensitive to the selection of hyperparameters, which we provide in the appendix. Code available at https://github.com/dylanrandle/denn. Please address any electronic correspondence to dylanrandle@alumni.harvard.edu.

preprint2019arXiv

An Information Theory Approach on Deciding Spectroscopic Follow Ups

Classification and characterization of variable phenomena and transient phenomena are critical for astrophysics and cosmology. These objects are commonly studied using photometric time series or spectroscopic data. Given that many ongoing and future surveys are in time-domain and given that adding spectra provide further insights but requires more observational resources, it would be valuable to know which objects should we prioritize to have spectrum in addition to time series. We propose a methodology in a probabilistic setting that determines a-priory which objects are worth taking spectrum to obtain better insights, where we focus 'insight' as the type of the object (classification). Objects for which we query its spectrum are reclassified using their full spectrum information. We first train two classifiers, one that uses photometric data and another that uses photometric and spectroscopic data together. Then for each photometric object we estimate the probability of each possible spectrum outcome. We combine these models in various probabilistic frameworks (strategies) which are used to guide the selection of follow up observations. The best strategy depends on the intended use, whether it is getting more confidence or accuracy. For a given number of candidate objects (127, equal to 5% of the dataset) for taking spectra, we improve 37% class prediction accuracy as opposed to 20% of a non-naive (non-random) best base-line strategy. Our approach provides a general framework for follow-up strategies and can be extended beyond classification and to include other forms of follow-ups beyond spectroscopy.

preprint2019arXiv

Streaming Classification of Variable Stars

In the last years, automatic classification of variable stars has received substantial attention. Using machine learning techniques for this task has proven to be quite useful. Typically, machine learning classifiers used for this task require to have a fixed training set, and the training process is performed offline. Upcoming surveys such as the Large Synoptic Survey Telescope (LSST) will generate new observations daily, where an automatic classification system able to create alerts online will be mandatory. A system with those characteristics must be able to update itself incrementally. Unfortunately, after training, most machine learning classifiers do not support the inclusion of new observations in light curves, they need to re-train from scratch. Naively re-training from scratch is not an option in streaming settings, mainly because of the expensive pre-processing routines required to obtain a vector representation of light curves (features) each time we include new observations. In this work, we propose a streaming probabilistic classification model; it uses a set of newly designed features that work incrementally. With this model, we can have a machine learning classifier that updates itself in real time with new observations. To test our approach, we simulate a streaming scenario with light curves from CoRot, OGLE and MACHO catalogs. Results show that our model achieves high classification performance, staying an order of magnitude faster than traditional classification approaches.

preprint2016arXiv

Clustering Based Feature Learning on Variable Stars

The success of automatic classification of variable stars strongly depends on the lightcurve representation. Usually, lightcurves are represented as a vector of many statistical descriptors designed by astronomers called features. These descriptors commonly demand significant computational power to calculate, require substantial research effort to develop and do not guarantee good performance on the final classification task. Today, lightcurve representation is not entirely automatic; algorithms that extract lightcurve features are designed by humans and must be manually tuned up for every survey. The vast amounts of data that will be generated in future surveys like LSST mean astronomers must develop analysis pipelines that are both scalable and automated. Recently, substantial efforts have been made in the machine learning community to develop methods that prescind from expert-designed and manually tuned features for features that are automatically learned from data. In this work we present what is, to our knowledge, the first unsupervised feature learning algorithm designed for variable stars. Our method first extracts a large number of lightcurve subsequences from a given set of photometric data, which are then clustered to find common local patterns in the time series. Representatives of these patterns, called exemplars, are then used to transform lightcurves of a labeled set into a new representation that can then be used to train an automatic classifier. The proposed algorithm learns the features from both labeled and unlabeled lightcurves, overcoming the bias generated when the learning process is done only with labeled data. We test our method on MACHO and OGLE datasets; the results show that the classification performance we achieve is as good and in some cases better than the performance achieved using traditional features, while the computational cost is significantly lower.

preprint2016arXiv

Meta Classification for Variable Stars

The need for the development of automatic tools to explore astronomical databases has been recognized since the inception of CCDs and modern computers. Astronomers already have developed solutions to tackle several science problems, such as automatic classification of stellar objects, outlier detection, and globular clusters identification, among others. New science problems emerge and it is critical to be able to re-use the models learned before, without rebuilding everything from the beginning when the science problem changes. In this paper, we propose a new meta-model that automatically integrates existing classification models of variable stars. The proposed meta-model incorporates existing models that are trained in a different context, answering different questions and using different representations of data. Conventional mixture of experts algorithms in machine learning literature can not be used since each expert (model) uses different inputs. We also consider computational complexity of the model by using the most expensive models only when it is necessary. We test our model with EROS-2 and MACHO datasets, and we show that we solve most of the classification challenges only by training a meta-model to learn how to integrate the previous experts.

preprint2015arXiv

Computational Intelligence Challenges and Applications on Large-Scale Astronomical Time Series Databases

Time-domain astronomy (TDA) is facing a paradigm shift caused by the exponential growth of the sample size, data complexity and data generation rates of new astronomical sky surveys. For example, the Large Synoptic Survey Telescope (LSST), which will begin operations in northern Chile in 2022, will generate a nearly 150 Petabyte imaging dataset of the southern hemisphere sky. The LSST will stream data at rates of 2 Terabytes per hour, effectively capturing an unprecedented movie of the sky. The LSST is expected not only to improve our understanding of time-varying astrophysical objects, but also to reveal a plethora of yet unknown faint and fast-varying phenomena. To cope with a change of paradigm to data-driven astronomy, the fields of astroinformatics and astrostatistics have been created recently. The new data-oriented paradigms for astronomy combine statistics, data mining, knowledge discovery, machine learning and computational intelligence, in order to provide the automated and robust methods needed for the rapid detection and classification of known astrophysical objects as well as the unsupervised characterization of novel phenomena. In this article we present an overview of machine learning and computational intelligence applications to TDA. Future big data challenges and new lines of research in TDA, focusing on the LSST, are identified and discussed from the viewpoint of computational intelligence/machine learning. Interdisciplinary collaboration will be required to cope with the challenges posed by the deluge of astronomical data coming from the LSST.

preprint2015arXiv

Fast and optimal nonparametric sequential design for astronomical observations

The spectral energy distribution (SED) is a relatively easy way for astronomers to distinguish between different astronomical objects such as galaxies, black holes, and stellar objects. By comparing the observations from a source at different frequencies with template models, astronomers are able to infer the type of this observed object. In this paper, we take a Bayesian model averaging perspective to learn astronomical objects, employing a Bayesian nonparametric approach to accommodate the deviation from convex combinations of known log-SEDs. To effectively use telescope time for observations, we then study Bayesian nonparametric sequential experimental design without conjugacy, in which we use sequential Monte Carlo as an efficient tool to maximize the volume of information stored in the posterior distribution of the parameters of interest. A new technique for performing inferences in log-Gaussian Cox processes called the Poisson log-normal approximation is also proposed. Simulations show the speed, accuracy, and usefulness of our method. While the strategy we propose in this paper is brand new in the astronomy literature, the inferential techniques developed apply to more general nonparametric sequential experimental design problems.

preprint2015arXiv

FATS: Feature Analysis for Time Series

In this paper, we present the FATS (Feature Analysis for Time Series) library. FATS is a Python library which facilitates and standardizes feature extraction for time series data. In particular, we focus on one application: feature extraction for astronomical light curve data, although the library is generalizable for other uses. We detail the methods and features implemented for light curve analysis, and present examples for its usage.

preprint2015arXiv

Supervised detection of anomalous light-curves in massive astronomical catalogs

The development of synoptic sky surveys has led to a massive amount of data for which resources needed for analysis are beyond human capabilities. To process this information and to extract all possible knowledge, machine learning techniques become necessary. Here we present a new method to automatically discover unknown variable objects in large astronomical catalogs. With the aim of taking full advantage of all the information we have about known objects, our method is based on a supervised algorithm. In particular, we train a random forest classifier using known variability classes of objects and obtain votes for each of the objects in the training set. We then model this voting distribution with a Bayesian network and obtain the joint voting distribution among the training objects. Consequently, an unknown object is considered as an outlier insofar it has a low joint probability. Our method is suitable for exploring massive datasets given that the training process is performed offline. We tested our algorithm on 20 millions light-curves from the MACHO catalog and generated a list of anomalous candidates. We divided the candidates into two main classes of outliers: artifacts and intrinsic outliers. Artifacts were principally due to air mass variation, seasonal variation, bad calibration or instrumental errors and were consequently removed from our outlier list and added to the training set. After retraining, we selected about 4000 objects, which we passed to a post analysis stage by perfoming a cross-match with all publicly available catalogs. Within these candidates we identified certain known but rare objects such as eclipsing Cepheids, blue variables, cataclysmic variables and X-ray sources. For some outliers there were no additional information. Among them we identified three unknown variability types and few individual outliers that will be followed up for a deeper analysis.

preprint2014arXiv

A Novel, Fully Automated Pipeline for Period Estimation in the EROS 2 Data Set

We present a new method to discriminate periodic from non-periodic irregularly sampled lightcurves. We introduce a periodic kernel and maximize a similarity measure derived from information theory to estimate the periods and a discriminator factor. We tested the method on a dataset containing 100,000 synthetic periodic and non-periodic lightcurves with various periods, amplitudes and shapes generated using a multivariate generative model. We correctly identified periodic and non-periodic lightcurves with a completeness of 90% and a precision of 95%, for lightcurves with a signal-to-noise ratio (SNR) larger than 0.5. We characterize the efficiency and reliability of the model using these synthetic lightcurves and applied the method on the EROS-2 dataset. A crucial consideration is the speed at which the method can be executed. Using hierarchical search and some simplification on the parameter search we were able to analyze 32.8 million lightcurves in 18 hours on a cluster of GPGPUs. Using the sensitivity analysis on the synthetic dataset, we infer that 0.42% in the LMC and 0.61% in the SMC of the sources show periodic behavior. The training set, the catalogs and source code are all available in http://timemachine.iic.harvard.edu.

preprint2014arXiv

The EPOCH Project: I. Periodic variable stars in the EROS-2 LMC database

The EPOCH (EROS-2 periodic variable star classification using machine learning) project aims to detect periodic variable stars in the EROS-2 light curve database. In this paper, we present the first result of the classification of periodic variable stars in the EROS-2 LMC database. To classify these variables, we first built a training set by compiling known variables in the Large Magellanic Cloud area from the OGLE and MACHO surveys. We crossmatched these variables with the EROS-2 sources and extracted 22 variability features from 28 392 light curves of the corresponding EROS-2 sources. We then used the random forest method to classify the EROS-2 sources in the training set. We designed the model to separate not only $δ$ Scuti stars, RR Lyraes, Cepheids, eclipsing binaries, and long-period variables, the superclasses, but also their subclasses, such as RRab, RRc, RRd, and RRe for RR Lyraes, and similarly for the other variable types. The model trained using only the superclasses shows 99% recall and precision, while the model trained on all subclasses shows 87% recall and precision. We applied the trained model to the entire EROS-2 LMC database, which contains about 29 million sources, and found 117 234 periodic variable candidates. Out of these 117 234 periodic variables, 55 285 have not been discovered by either OGLE or MACHO variability studies. This set comprises 1 906 $δ$ Scuti stars, 6 607 RR Lyraes, 638 Cepheids, 178 Type II Cepheids, 34 562 eclipsing binaries, and 11 394 long-period variables. A catalog of these EROS-2 LMC periodic variable stars will be available online at http://stardb.yonsei.ac.kr and at the CDS website (http://vizier.u-strasbg.fr/viz-bin/VizieR).

preprint2014arXiv

The expansion rate of the intermediate Universe in light of Planck

We use cosmology-independent measurements of the expansion history in the redshift range 0.1 < z <1.2 and compare them with the Cosmic Microwave Background-derived expansion history predictions. The motivation is to investigate if the tension between the local (cosmology independent) Hubble constant H0 value and the Planck-derived H0 is also present at other redshifts. We conclude that there is no tension between Planck and cosmology independent-measurements of the Hubble parameter H(z) at 0.1 < z < 1.2 for the LCDM model (odds of tension are only 1:15, statistically not significant). Considering extensions of the LCDM model does not improve these odds (actually makes them worse), thus favouring the simpler model over its extensions. On the other hand the H(z) data are also not in tension with the local H0 measurements but the combination of all three data-sets shows a highly significant tension (odds ~ 1:400). Thus the new data deepen the mystery of the mismatch between Planck and local H0 measurements, and cannot univocally determine wether it is an effect localised at a particular redshift. Having said this, we find that assuming the NGC4258 maser distance as the correct anchor for H0, brings the odds to comfortable values. Further, using only the expansion history measurements we constrain, within the LCDM model, H0 = 68.5 +- 3.5 and Omega_m = 0.32 +- 0.05 without relying on any CMB prior. We also address the question of how smooth the expansion history of the universe is given the cosmology independent data and conclude that there is no evidence for deviations from smoothness on the expansion history, neither variations with time in the value of the equation of state of dark energy.

preprint2013arXiv

Automatic Classification of Variable Stars in Catalogs with missing data

We present an automatic classification method for astronomical catalogs with missing data. We use Bayesian networks, a probabilistic graphical model, that allows us to perform inference to pre- dict missing values given observed data and dependency relationships between variables. To learn a Bayesian network from incomplete data, we use an iterative algorithm that utilises sampling methods and expectation maximization to estimate the distributions and probabilistic dependencies of variables from data with missing values. To test our model we use three catalogs with missing data (SAGE, 2MASS and UBVI) and one complete catalog (MACHO). We examine how classification accuracy changes when information from missing data catalogs is included, how our method compares to traditional missing data approaches and at what computational cost. Integrating these catalogs with missing data we find that classification of variable objects improves by few percent and by 15% for quasar detection while keeping the computational cost the same.

preprint2013arXiv

Infinite Shift-invariant Grouped Multi-task Learning for Gaussian Processes

Multi-task learning leverages shared information among data sets to improve the learning performance of individual tasks. The paper applies this framework for data where each task is a phase-shifted periodic time series. In particular, we develop a novel Bayesian nonparametric model capturing a mixture of Gaussian processes where each task is a sum of a group-specific function and a component capturing individual variation, in addition to each task being phase shifted. We develop an efficient \textsc{em} algorithm to learn the parameters of the model. As a special case we obtain the Gaussian mixture model and \textsc{em} algorithm for phased-shifted periodic time series. Furthermore, we extend the proposed model by using a Dirichlet Process prior and thereby leading to an infinite mixture model that is capable of doing automatic model selection. A Variational Bayesian approach is developed for inference in this model. Experiments in regression, classification and class discovery demonstrate the performance of the proposed models using both synthetic data and real-world time series data from astrophysics. Our methods are particularly useful when the time series are sparsely and non-synchronously sampled.

preprint2013arXiv

Planck and the local Universe: quantifying the tension

We use the latest Planck constraints, and in particular constraints on the derived parameters (Hubble constant and age of the Universe) for the local universe and compare them with local measurements of the same quantities. We propose a way to quantify whether cosmological parameters constraints from two different experiments are in tension or not. Our statistic, T, is an evidence ratio and therefore can be interpreted with the widely used Jeffrey's scale. We find that in the framework of the LCDM model, the Planck inferred two dimensional, joint, posterior distribution for the Hubble constant and age of the Universe is in "strong" tension with the local measurements; the odds being ~ 1:50. We explore several possibilities for explaining this tension and examine the consequences both in terms of unknown errors and deviations from the LCDM model. In some one-parameter LCDM model extensions, tension is reduced whereas in other extensions, tension is instead increased. In particular, small total neutrino masses are favored and a total neutrino mass above 0.15 eV makes the tension "highly significant" (odds ~ 1:150). A consequence of accepting this interpretation of the tension is that the degenerate neutrino hierarchy is highly disfavoured by cosmological data and the direct hierarchy is slightly favored over the inverse.

preprint2013arXiv

Semi-parametric Robust Event Detection for Massive Time-Domain Databases

The detection and analysis of events within massive collections of time-series has become an extremely important task for time-domain astronomy. In particular, many scientific investigations (e.g. the analysis of microlensing and other transients) begin with the detection of isolated events in irregularly-sampled series with both non-linear trends and non-Gaussian noise. We outline a semi-parametric, robust, parallel method for identifying variability and isolated events at multiple scales in the presence of the above complications. This approach harnesses the power of Bayesian modeling while maintaining much of the speed and scalability of more ad-hoc machine learning approaches. We also contrast this work with event detection methods from other fields, highlighting the unique challenges posed by astronomical surveys. Finally, we present results from the application of this method to 87.2 million EROS-2 sources, where we have obtained a greater than 100-fold reduction in candidates for certain types of phenomena while creating high-quality features for subsequent analyses.

preprint2013arXiv

Statistical Properties of Galactic δ Scuti Stars: Revisited

We present statistical characteristics of 1,578 δ Scuti stars including nearby field stars and cluster member stars within the Milky Way. We obtained 46% of these stars (718 stars) from the works done by Rodríguez and collected the remaining 54% stars (860 stars) from other literatures. We updated the entries with the latest information of sky coordinate, color, rotational velocity, spectral type, period, amplitude and binarity. The majority of our sample are well characterized in terms of typical period range (0.02-0.25 days), pulsation amplitudes (<0.5 mag) and spectral types (A-F type). Given this list of δ Scuti stars, we examined relations between their physical properties (i.e., periods, amplitudes, spectral types and rotational velocities) for field stars and cluster members, and confirmed that the correlations of properties are not significantly different from those reported in the Rodríguez's works. All the δ Scuti stars are cross-matched with several X-ray and UV catalogs, resulting in 27 X-ray and 41 UV-only counterparts. These counterparts are interesting targets for further study because of their rarity and uniqueness in showing δ Scuti-type variability and X-ray/UV emission at the same time. The compiled catalog can be accessed through the web interface http://stardb.yonsei.ac.kr/DeltaScuti

preprint2012arXiv

An Information Theoretic Algorithm for Finding Periodicities in Stellar Light Curves

We propose a new information theoretic metric for finding periodicities in stellar light curves. Light curves are astronomical time series of brightness over time, and are characterized as being noisy and unevenly sampled. The proposed metric combines correntropy (generalized correlation) with a periodic kernel to measure similarity among samples separated by a given period. The new metric provides a periodogram, called Correntropy Kernelized Periodogram (CKP), whose peaks are associated with the fundamental frequencies present in the data. The CKP does not require any resampling, slotting or folding scheme as it is computed directly from the available samples. CKP is the main part of a fully-automated pipeline for periodic light curve discrimination to be used in astronomical survey databases. We show that the CKP method outperformed the slotted correntropy, and conventional methods used in astronomy for periodicity discrimination and period estimation tasks, using a set of light curves drawn from the MACHO survey. The proposed metric achieved 97.2% of true positives with 0% of false positives at the confidence level of 99% for the periodicity discrimination task; and 88% of hits with 11.6% of multiples and 0.4% of misses in the period estimation task.

preprint2012arXiv

Nonparametric Bayesian Estimation of Periodic Functions

Many real world problems exhibit patterns that have periodic behavior. For example, in astrophysics, periodic variable stars play a pivotal role in understanding our universe. An important step when analyzing data from such processes is the problem of identifying the period: estimating the period of a periodic function based on noisy observations made at irregularly spaced time points. This problem is still a difficult challenge despite extensive study in different disciplines. The paper makes several contributions toward solving this problem. First, we present a nonparametric Bayesian model for period finding, based on Gaussian Processes (GP), that does not make strong assumptions on the shape of the periodic function. As our experiments demonstrate, the new model leads to significantly better results in period estimation when the target function is non-sinusoidal. Second, we develop a new algorithm for parameter optimization for GP which is useful when the likelihood function is very sensitive to the setting of the hyper-parameters with numerous local minima, as in the case of period estimation. The algorithm combines gradient optimization with grid search and incorporates several mechanisms to overcome the high complexity of inference with GP. Third, we develop a novel approach for using domain knowledge, in the form of a probabilistic generative model, and incorporate it into the period estimation algorithm. Experimental results on astrophysics data validate our approach showing significant improvement over the state of the art in this domain.

preprint2011arXiv

A Refined QSO Selection Method Using Diagnostics Tests: 663 QSO Candidates in the LMC

We present 663 QSO candidates in the Large Magellanic Cloud (LMC) selected using multiple diagnostics. We started with a set of 2,566 QSO candidates from our previous work selected using time variability of the MACHO LMC lightcurves. We then obtained additional information for the candidates by crossmatching them with the Spitzer SAGE, the MACHO UBVI, the 2MASS, the Chandra and the XMM catalogs. Using this information, we specified six diagnostic features based on mid-IR colors, photometric redshifts using SED template fitting, and X-ray luminosities in order to further discriminate high confidence QSO candidates in the absence of spectra information. We then trained a one-class SVM (Support Vector Machine) model using the diagnostics features of the confirmed 58 MACHO QSOs. We applied the trained model to the original candidates and finally selected 663 high confidence QSO candidates. Furthermore, we crossmatched these 663 QSO candidates with the newly confirmed 144 QSOs and 275 non-QSOs in the LMC fields. On the basis of the counterpart analysis, we found that the false positive rate is less than 1%.

preprint2011arXiv

Period Estimation in Astronomical Time Series Using Slotted Correntropy

In this letter, we propose a method for period estimation in light curves from periodic variable stars using correntropy. Light curves are astronomical time series of stellar brightness over time, and are characterized as being noisy and unevenly sampled. We propose to use slotted time lags in order to estimate correntropy directly from irregularly sampled time series. A new information theoretic metric is proposed for discriminating among the peaks of the correntropy spectral density. The slotted correntropy method outperformed slotted correlation, string length, VarTools (Lomb-Scargle periodogram and Analysis of Variance), and SigSpec applications on a set of light curves drawn from the MACHO survey.

preprint2011arXiv

QSO Selection Algorithm Using Time Variability and Machine Learning: Selection of 1,620 QSO Candidates from MACHO LMC Database

We present a new QSO selection algorithm using a Support Vector Machine (SVM), a supervised classification method, on a set of extracted times series features including period, amplitude, color, and autocorrelation value. We train a model that separates QSOs from variable stars, non-variable stars and microlensing events using 58 known QSOs, 1,629 variable stars and 4,288 non-variables using the MAssive Compact Halo Object (MACHO) database as a training set. To estimate the efficiency and the accuracy of the model, we perform a cross-validation test using the training set. The test shows that the model correctly identifies ~80% of known QSOs with a 25% false positive rate. The majority of the false positives are Be stars. We applied the trained model to the MACHO Large Magellanic Cloud (LMC) dataset, which consists of 40 million lightcurves, and found 1,620 QSO candidates. During the selection none of the 33,242 known MACHO variables were misclassified as QSO candidates. In order to estimate the true false positive rate, we crossmatched the candidates with astronomical catalogs including the Spitzer Surveying the Agents of a Galaxy's Evolution (SAGE) LMC catalog and a few X-ray catalogs. The results further suggest that the majority of the candidates, more than 70%, are QSOs.

preprint2010arXiv

Trans-Neptunian Objects with Hubble Space Telescope ACS/WFC

We introduce a novel search technique that can identify trans-neptunian objects in three to five exposures of a pointing within a single Hubble Space Telescope orbit. The process is fast enough to allow the discovery of candidates soon after the data are available. This allows sufficient time to schedule follow up observations with HST within a month. We report the discovery of 14 slow-moving objects found within 5\circ of the ecliptic in archival data taken with the Wide Field Channel of the Advanced Camera for Surveys. The luminosity function of these objects is consistent with previous ground-based and space-based results. We show evidence that the size distribution of both high and low inclination populations is similar for objects smaller than 100 km, as expected from collisional evolution models, while their size distribution differ for brighter objects. We suggest the two populations formed in different parts of the protoplanetary disk and after being dynamically mixed have collisionally evolved together. Among the objects discovered there is an equal mass binary with an angular separation ~ 0."53.

preprint2009arXiv

A Bayesian approach to the analysis of time symmetry in light curves: Reconsidering Scorpius X-1 occultations

We present a new approach to the analysis of time symmetry in light curves, such as those in the x-ray at the center of the Scorpius X-1 occultation debate. Our method uses a new parameterization for such events (the bilogistic event profile) and provides a clear, physically relevant characterization of each event's key features. We also demonstrate a Markov Chain Monte Carlo algorithm to carry out this analysis, including a novel independence chain configuration for the estimation of each event's location in the light curve. These tools are applied to the Scorpius X-1 light curves presented in Chang et al. (2007), providing additional evidence based on the time series that the events detected thus far are most likely not occultations by TNOs.

preprint2009arXiv

A Search for Occultations of Bright Stars by Small Kuiper Belt Objects using Megacam on the MMT

We conducted a search for occultations of bright stars by Kuiper Belt Objects (KBOs) to estimate the density of sub-km KBOs in the sky. We report here the first results of this occultation survey of the outer solar system conducted in June 2007 and June/July 2008 at the MMT Observatory using Megacam, the large MMT optical imager. We used Megacam in a novel shutterless continuous--readout mode to achieve high precision photometry at 200 Hz. We present an analysis of 220 star hours at signal-to-noise ratio of 25 or greater. The survey efficiency is greater than 10% for occultations by KBOs of diameter d>=0.7 km, and we report no detections in our dataset. We set a new 95% confidence level upper limit for the surface density Σ_N(d) of KBOs larger than 1 km: Σ_N(d>=1 km) <= 2.0e8 deg^-2, and for KBOs larger than 0.7 km Σ_N(d>= 0.7 km) <= 4.8e8 deg^-2.

preprint2009arXiv

Reverberation in the UV-Optical Continuum Brightness Fluctuations of MACHO Quasar 13.5962.237

We examine the nature of brightness fluctuations in the UV-Optical spectral region of an ordinary quasar with 881 optical brightness measurements made during the epoch 1993 - 1999. We find evidence for systematic trends having the character of a pattern of reverberations following an initial disturbance. The initial pulses have brightness increases of order 20% and pulse widths of 50 days, and the reverberations have typical amplitudes of 12% with longer mean pulse widths of order 80 days and pulse separations of order 90 days. The repeat pattern occurs over the same time scales whether the initial disturbance is a brightening or fading. The lags of the pulse trains are comparable to the lags seen previously in reverberation of the broad blue-shifted emission lines following brightness disturbances in Seyfert galaxies, when allowance is made for the mass of the central object. In addition to the burst pulse trains, we find evidence for a semi-periodicity with a time scale of 2 years. These strong patterns of brightness fluctuations suggest a method of discovering quasars from photometric monitoring alone, with data of the quality expected from large brightness monitoring programs like Pann-Stars and LSST.

preprint2007arXiv

Eclipsing binary stars in the Large and Small Magellanic Clouds from the MACHO project: The Sample

We present a new sample of 4634 eclipsing binary stars in the Large Magellanic Cloud (LMC), expanding on a previous sample of 611 objects and a new sample of 1509 eclipsing binary stars in the Small Magellanic Cloud (SMC), that were identified in the light curve database of the MACHO project. We perform a cross correlation with the OGLE-II LMC sample, finding 1236 matches. A cross correlation with the OGLE-II SMC sample finds 698 matches. We then compare the LMC subsamples corresponding to center and the periphery of the LMC and find only minor differences between the two populations. These samples are sufficiently large and complete that statistical studies of the binary star populations are possible.

preprint1996arXiv

Possible solution of the Coriolis attenuation problem

The most consistently useful simple model for the study of odd deformed nuclei, the particle-rotor model (strong coupling limit of the core-particle coupling model) has nevertheless been beset by a long-standing problem: It is necessary in many cases to introduce an ad hoc parameter that reduces the size of the Coriolis interaction coupling the collective and single-particle motions. Of the numerous suggestions put forward for the origin of this supplementary interaction, none of those actually tested by calculations has been accepted as the solution of the problem. In this paper we seek a solution of the difficulty within the framework of a general formalism that starts from the spherical shell model and is capable of treating an arbitrary linear combination of multipole and pairing forces. With the restriction of the interaction to the familiar sum of a quadrupole multipole force and a monopole pairing force, we have previously studied a semi-microscopic version of the formalism whose framework is nevertheless more comprehensive than any previously applied to the problem. We obtained solutions for low-lying bands of several strongly deformed odd rare earth nuclei and found good agreement with experiment, except for an exaggerated staggering of levels for K=1/2 bands, which can be understood as a manifestation of the Coriolis attenuation problem. We argue that within the formalism utilized, the only way to improve the physics is to add interactions to the model Hamiltonian. We verify that by adding a magnetic dipole interaction of essentially fixed strength, we can fit the K=1/2 bands without destroying the agreement with other bands. In addition we show that our solution also fits 163Er, a classic test case of Coriolis attenuation that we had not previously studied.