Source author record

Isaac Tamblyn

Isaac Tamblyn appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

Catalog footprint

What is connected

27works
16topics
4close collaborators

Actions

Connect this record

Log in to claim

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

27 published item(s)

preprint2023arXiv

fintech-kMC: Agent based simulations of financial platforms for design and testing of machine learning systems

We discuss our simulation tool, fintech-kMC, which is designed to generate synthetic data for machine learning model development and testing. fintech-kMC is an agent-based model driven by a kinetic Monte Carlo (a.k.a. continuous time Monte Carlo) engine which simulates the behaviour of customers using an online digital financial platform. The tool provides an interpretable, reproducible, and realistic way of generating synthetic data which can be used to validate and test AI/ML models and pipelines to be used in real-world customer-facing financial applications.

preprint2022arXiv

Dynamic programming with incomplete information to overcome navigational uncertainty in a nautical environment

Using a novel toy nautical navigation environment, we show that dynamic programming can be used when only incomplete information about a partially observed Markov decision process (POMDP) is known. By incorporating uncertainty into our model, we show that navigation policies can be constructed that maintain safety, outperforming the baseline performance of traditional dynamic programming for Markov decision processes (MDPs). Adding in controlled sensing methods, we show that these policies can also lower measurement costs at the same time.

preprint2022arXiv

Generative Enriched Sequential Learning (ESL) Approach for Molecular Design via Augmented Domain Knowledge

Deploying generative machine learning techniques to generate novel chemical structures based on molecular fingerprint representation has been well established in molecular design. Typically, sequential learning (SL) schemes such as hidden Markov models (HMM) and, more recently, in the sequential deep learning context, recurrent neural network (RNN) and long short-term memory (LSTM) were used extensively as generative models to discover unprecedented molecules. To this end, emission probability between two states of atoms plays a central role without considering specific chemical or physical properties. Lack of supervised domain knowledge can mislead the learning procedure to be relatively biased to the prevalent molecules observed in the training data that are not necessarily of interest. We alleviated this drawback by augmenting the training data with domain knowledge, e.g. quantitative estimates of the drug-likeness score (QEDs). As such, our experiments demonstrated that with this subtle trick called enriched sequential learning (ESL), specific patterns of particular interest can be learnt better, which led to generating de novo molecules with ameliorated QEDs.

preprint2022arXiv

Learning stochastic dynamics and predicting emergent behavior using transformers

We show that a neural network originally designed for language processing can learn the dynamical rules of a stochastic system by observation of a single dynamical trajectory of the system, and can accurately predict its emergent behavior under conditions not observed during training. We consider a lattice model of active matter undergoing continuous-time Monte Carlo dynamics, simulated at a density at which its steady state comprises small, dispersed clusters. We train a neural network called a transformer on a single trajectory of the model. The transformer, which we show has the capacity to represent dynamical rules that are numerous and nonlocal, learns that the dynamics of this model consists of a small number of processes. Forward-propagated trajectories of the trained transformer, at densities not encountered during training, exhibit motility-induced phase separation and so predict the existence of a nonequilibrium phase transition. Transformers have the flexibility to learn dynamical rules from observation without explicit enumeration of rates or coarse-graining of configuration space, and so the procedure used here can be applied to a wide range of physical systems, including those with large and complex dynamical generators.

preprint2022arXiv

Scientific Discovery and the Cost of Measurement -- Balancing Information and Cost in Reinforcement Learning

The use of reinforcement learning (RL) in scientific applications, such as materials design and automated chemistry, is increasing. A major challenge, however, lies in fact that measuring the state of the system is often costly and time consuming in scientific applications, whereas policy learning with RL requires a measurement after each time step. In this work, we make the measurement costs explicit in the form of a costed reward and propose a framework that enables off-the-shelf deep RL algorithms to learn a policy for both selecting actions and determining whether or not to measure the current state of the system at each time step. In this way, the agents learn to balance the need for information with the cost of information. Our results show that when trained under this regime, the Dueling DQN and PPO agents can learn optimal action policies whilst making up to 50\% fewer state measurements, and recurrent neural networks can produce a greater than 50\% reduction in measurements. We postulate the these reduction can help to lower the barrier to applying RL to real-world scientific applications.

preprint2022arXiv

Toward Orbital-Free Density Functional Theory with Small Data Sets and Deep Learning

We use voxel deep neural networks to predict energy densities and functional derivatives of electron kinetic energies for the Thomas-Fermi model and Kohn-Sham density functional theory calculations. We show that the ground-state electron density can be found via direct minimization for a graphene lattice without any projection scheme using a voxel deep neural network trained with the Thomas-Fermi model. Additionally, we predict the kinetic energy of a graphene lattice within chemical accuracy after training from only 2 Kohn-Sham density functional theory (DFT) calculations. We identify an important sampling issue inherent in Kohn-Sham DFT calculations and propose future work to rectify this problem. Furthermore, we demonstrate an alternative, functional derivative-free, Monte Carlo based orbital free density functional theory algorithm to calculate an accurate 2-electron density in a double inverted Gaussian potential with a machine-learned kinetic energy functional.

preprint2022arXiv

Training neural networks using Metropolis Monte Carlo and an adaptive variant

We examine the zero-temperature Metropolis Monte Carlo algorithm as a tool for training a neural network by minimizing a loss function. We find that, as expected on theoretical grounds and shown empirically by other authors, Metropolis Monte Carlo can train a neural net with an accuracy comparable to that of gradient descent, if not necessarily as quickly. The Metropolis algorithm does not fail automatically when the number of parameters of a neural network is large. It can fail when a neural network's structure or neuron activations are strongly heterogenous, and we introduce an adaptive Monte Carlo algorithm, aMC, to overcome these limitations. The intrinsic stochasticity and numerical stability of the Monte Carlo method allow aMC to train deep neural networks and recurrent neural networks in which the gradient is too small or too large to allow training by gradient descent. Monte Carlo methods offer a complement to gradient-based methods for training neural networks, allowing access to a distinct set of network architectures and principles.

preprint2021arXiv

Controlled Online Optimization Learning (COOL): Finding the ground state of spin Hamiltonians with reinforcement learning

Reinforcement learning (RL) has become a proven method for optimizing a procedure for which success has been defined, but the specific actions needed to achieve it have not. We apply the so-called "black box" method of RL to what has been referred as the "black art" of simulated annealing (SA), demonstrating that an RL agent based on proximal policy optimization can, through experience alone, arrive at a temperature schedule that surpasses the performance of standard heuristic temperature schedules for two classes of Hamiltonians. When the system is initialized at a cool temperature, the RL agent learns to heat the system to "melt" it, and then slowly cool it in an effort to anneal to the ground state; if the system is initialized at a high temperature, the algorithm immediately cools the system. We investigate the performance of our RL-driven SA agent in generalizing to all Hamiltonians of a specific class; when trained on random Hamiltonians of nearest-neighbour spin glasses, the RL agent is able to control the SA process for other Hamiltonians, reaching the ground state with a higher probability than a simple linear annealing schedule. Furthermore, the scaling performance (with respect to system size) of the RL approach is far more favourable, achieving a performance improvement of one order of magnitude on L=14x14 systems. We demonstrate the robustness of the RL approach when the system operates in a "destructive observation" mode, an allusion to a quantum system where measurements destroy the state of the system. The success of the RL agent could have far-reaching impact, from classical optimization, to quantum annealing, to the simulation of physical systems.

preprint2021arXiv

Correspondence between neuroevolution and gradient descent

We show analytically that training a neural network by conditioned stochastic mutation or neuroevolution of its weights is equivalent, in the limit of small mutations, to gradient descent on the loss function in the presence of Gaussian white noise. Averaged over independent realizations of the learning process, neuroevolution is equivalent to gradient descent on the loss function. We use numerical simulation to show that this correspondence can be observed for finite mutations,for shallow and deep neural networks. Our results provide a connection between two families of neural-network training methods that are usually considered to be fundamentally different.

preprint2021arXiv

Deep Learning and Density Functional Theory

We show that deep neural networks can be integrated into, or fully replace, the Kohn-Sham density functional theory scheme for multi-electron systems in simple harmonic oscillator and random external potentials with no feature engineering. We first show that self-consistent charge densities calculated with different exchange-correlation functionals can be used as input to an extensive deep neural network to make predictions for correlation, exchange, external, kinetic and total energies simultaneously. Additionally, we show that one can also make all of the same predictions with the external potential rather than the self-consistent charge density, which allows one to circumvent the Kohn-Sham scheme altogether. We then show that a self-consistent charge density found from a non-local exchange-correlation functional can be used to make energy predictions for a semi-local exchange-correlation functional. Lastly, we use a deep convolutional inverse graphics network to predict the charge density given an external potential for different exchange-correlation functionals and assess the viability of the predicted charge densities. This work shows that extensive deep neural networks are generalizable and transferable given the variability of the potentials (maximum total energy range $\approx100$ Ha), because they require no feature engineering, and because they can scale to an arbitrary system size with an $\mathcal{O}(N)$ computational cost.

preprint2021arXiv

Interpretable discovery of new semiconductors with machine learning

Machine learning models of materials$^{1-5}$ accelerate discovery compared to ab initio methods: deep learning models now reproduce density functional theory (DFT)-calculated results at one hundred thousandths of the cost of DFT$^{6}$. To provide guidance in experimental materials synthesis, these need to be coupled with an accurate yet effective search algorithm and training data consistent with experimental observations. Here we report an evolutionary algorithm powered search which uses machine-learned surrogate models trained on high-throughput hybrid functional DFT data benchmarked against experimental bandgaps: Deep Adaptive Regressive Weighted Intelligent Network (DARWIN). The strategy enables efficient search over the materials space of ~10$^8$ ternaries and 10$^{11}$ quaternaries$^{7}$ for candidates with target properties. It provides interpretable design rules, such as our finding that the difference in the electronegativity between the halide and B-site cation being a strong predictor of ternary structural stability. As an example, when we seek UV emission, DARWIN predicts K$_2$CuX$_3$ (X = Cl, Br) as a promising materials family, based on its electronegativity difference. We synthesized and found these materials to be stable, direct bandgap UV emitters. The approach also allows knowledge distillation for use by humans.

preprint2021arXiv

Inverse Design of a Graphene-Based Quantum Transducer via Neuroevolution

We introduce an inverse design framework based on artificial neural networks, genetic algorithms, and tight-binding calculations, capable to optimize the very large configuration space of nanoelectronic devices. Our non-linear optimization procedure operates on trial Hamiltonians through superoperators controlling growth policies of regions of distinct doping. We demonstrate that our algorithm optimizes the doping of graphene-based three-terminal devices for valleytronics applications, monotonously converging to synthesizable devices with high merit functions in a few thousand evaluations (out of $\simeq 2^{3800}$ possible configurations). The best-performing device allowed for a terminal-specific separation of valley currents with $\simeq 96$\% ($\simeq 94\%)$ $K$ ($K'$) valley purity. Importantly, the devices found through our non-linear optimization procedure have both higher merit function and higher robustness to defects than the ones obtained through geometry optimization.

preprint2021arXiv

Weakly-supervised multi-class object localization using only object counts as labels

We demonstrate the use of an extensive deep neural network to localize instances of objects in images. The EDNN is naturally able to accurately perform multi-class counting using only ground truth count values as labels. Without providing any conceptual information, object annotations, or pixel segmentation information, the neural network is able to formulate its own conceptual representation of the items in the image. Using images labelled with only the counts of the objects present,the structure of the extensive deep neural network can be exploited to perform localization of the objects within the visual field. We demonstrate that a trained EDNN can be used to count objects in images much larger than those on which it was trained. In order to demonstrate our technique, we introduce seven new data sets: five progressively harder MNIST digit-counting data sets, and two datasets of 3d-rendered rubber ducks in various situations. On most of these datasets, the EDNN achieves greater than 99% test set accuracy in counting objects.

preprint2020arXiv

Active Measure Reinforcement Learning for Observation Cost Minimization

Standard reinforcement learning (RL) algorithms assume that the observation of the next state comes instantaneously and at no cost. In a wide variety of sequential decision making tasks ranging from medical treatment to scientific discovery, however, multiple classes of state observations are possible, each of which has an associated cost. We propose the active measure RL framework (Amrl) as an initial solution to this problem where the agent learns to maximize the costed return, which we define as the discounted sum of rewards minus the sum of observation costs. Our empirical evaluation demonstrates that Amrl-Q agents are able to learn a policy and state estimator in parallel during online training. During training the agent naturally shifts from its reliance on costly measurements of the environment to its state estimator in order to increase its reward. It does this without harm to the learned policy. Our results show that the Amrl-Q agent learns at a rate similar to standard Q-learning and Dyna-Q. Critically, by utilizing an active strategy, Amrl-Q achieves a higher costed return.

preprint2020arXiv

Evolutionary reinforcement learning of dynamical large deviations

We show how to calculate the likelihood of dynamical large deviations using evolutionary reinforcement learning. An agent, a stochastic model, propagates a continuous-time Monte Carlo trajectory and receives a reward conditioned upon the values of certain path-extensive quantities. Evolution produces progressively fitter agents, eventually allowing the calculation of a piece of a large-deviation rate function for a particular model and path-extensive quantity. For models with small state spaces the evolutionary process acts directly on rates, and for models with large state spaces the process acts on the weights of a neural network that parameterizes the model's rates. This approach shows how path-extensive physics problems can be considered within a framework widely used in machine learning.

preprint2020arXiv

Learning to grow: control of material self-assembly using evolutionary reinforcement learning

We show that neural networks trained by evolutionary reinforcement learning can enact efficient molecular self-assembly protocols. Presented with molecular simulation trajectories, networks learn to change temperature and chemical potential in order to promote the assembly of desired structures or choose between competing polymorphs. In the first case, networks reproduce in a qualitative sense the results of previously-known protocols, but faster and with higher fidelity; in the second case they identify strategies previously unknown, from which we can extract physical insight. Networks that take as input the elapsed time of the simulation or microscopic information from the system are both effective, the latter more so. The evolutionary scheme we have used is simple to implement and can be applied to a broad range of examples of experimental self-assembly, whether or not one can monitor the experiment as it proceeds. Our results have been achieved with no human input beyond the specification of which order parameter to promote, pointing the way to the design of synthesis protocols by artificial intelligence.

preprint2020arXiv

Reinforcement Learning in a Physics-Inspired Semi-Markov Environment

Reinforcement learning (RL) has been demonstrated to have great potential in many applications of scientific discovery and design. Recent work includes, for example, the design of new structures and compositions of molecules for therapeutic drugs. Much of the existing work related to the application of RL to scientific domains, however, assumes that the available state representation obeys the Markov property. For reasons associated with time, cost, sensor accuracy, and gaps in scientific knowledge, many scientific design and discovery problems do not satisfy the Markov property. Thus, something other than a Markov decision process (MDP) should be used to plan / find the optimal policy. In this paper, we present a physics-inspired semi-Markov RL environment, namely the phase change environment. In addition, we evaluate the performance of value-based RL algorithms for both MDPs and partially observable MDPs (POMDPs) on the proposed environment. Our results demonstrate deep recurrent Q-networks (DRQN) significantly outperform deep Q-networks (DQN), and that DRQNs benefit from training with hindsight experience replay. Implications for the use of semi-Markovian RL and POMDPs for scientific laboratories are also discussed.

preprint2014arXiv

Common physical framework explains phase behavior and dynamics of atomic, molecular and polymeric network-formers

We show that the self-assembly of a diverse collection of building blocks can be understood within a common physical framework. These building blocks, which form periodic honeycomb networks and nonperiodic variants thereof, range in size from atoms to micron-scale polymers, and interact through mechanisms as different as hydrogen bonds and covalent forces. A combination of statistical mechanics and quantum mechanics shows that one can capture the physics that governs assembly of these networks by resolving only the geometry and strength of building block interactions. The resulting framework reproduces a broad range of phenomena seen experimentally, including periodic and nonperiodic networks in thermal equilibrium, and nonperiodic supercooled and glassy networks away from equilibrium. Our results show how simple `design criteria' control assembly of a wide variety of networks, and suggest that kinetic trapping can be a useful way of making functional assemblies.

preprint2014arXiv

Electronic Structure of Liquid Water and a Platinum Surface

Many-body perturbation theory within the G$_0$W$_0$ approximation is used to determine molecular orbital level alignment at a liquid water/Pt(111) interface generated through $ab~ initio$ molecular dynamics. Molecular orbital energy levels are shown to depend both on the position of H$_2$O molecules within the liquid relative to the surface and the details of their local bonding environment. Standard density functional theory calculations disagree qualitatively with level alignment predicted by many-body perturbation theory.

preprint2014arXiv

Emergent rhombus tilings from molecular interactions with $M$-fold rotational symmetry

We show that model molecules with particular rotational symmetries can self-assemble into network structures equivalent to rhombus tilings. This assembly happens in an emergent way, in the sense that molecules spontaneously select irregular 4-fold local coordination from a larger set of possible local binding geometries. The existence of such networks can be rationalized by simple geometrical arguments, but the same arguments do not guarantee networks' spontaneous self-assembly. This class of structures must in certain regimes of parameter space be able to reconfigure into networks equivalent to triangular tilings.

preprint2013arXiv

Competing thermodynamic and dynamic factors select molecular assemblies on a gold surface

Controlling the self-assembly of surface-adsorbed molecules into nanostructures requires understanding physical mechanisms that act across multiple length and time scales. By combining scanning tunneling microscopy with hierarchical ab initio and statistical mechanical modeling of 1,4-substituted benzenediamine (BDA) molecules adsorbed on a gold (111) surface, we demonstrate that apparently simple nanostructures are selected by a subtle competition of thermodynamics and dynamics. Of the collection of possible BDA nanostructures mechanically stabilized by hydrogen bonding, the interplay of intermolecular forces, surface modulation, and assembly dynamics select at low temperature a particular subset: low free energy oriented linear chains of monomers, and high free energy branched chains.

preprint2012arXiv

Molecular Adsorption on Metal Surfaces with a van der Waals Density Functional

The adsorption of 1,4-benzenediamine (BDA) on the Au(111) surface and azobenzene on the Ag(111) surface is investigated using density functional theory (DFT) with a non-local density functional (vdW-DF) and a semi-local Perdew-Burke-Ernzerhof (PBE) functional. For BDA on Au(111), the inclusion of London dispersion interactions not only dramatically enhances the molecule-substrate binding, resulting in adsorption energies consistent with experimental results, but also significantly alters the BDA binding geometry. For azobenzene on Ag(111), the vdW-DF produces superior adsorption energies compared to those obtained with other dispersion corrected DFT approaches. These results provide evidence for the applicability of the vdW-DF method and serves as a practical benchmark for the investigation of molecules adsorbed on noble metal surfaces.

preprint2012arXiv

Quantitative Molecular Orbital Energies within a $G_0W_0$ Approximation

Using many-body perturbation theory within the $G_0W_0$ approximation, we explore routes for computing the ionization potential (IP), electron affinity (EA), and fundamental gap of three gas-phase molecules -- benzene, thiophene, and (1,4) diamino-benzene -- and compare with experiments. We examine the dependence of the IP on the number of unoccupied states used to build the dielectric function and the self energy, as well as the dielectric function plane-wave cutoff. We find that with an effective completion strategy for approximating the unoccupied subspace, and a converged dielectric function kinetic energy cutoff, the computed IPs and EAs are in excellent quantitative agreement with available experiment (within 0.2 eV), indicating that a one-shot $G_0W_0$ approach can be very accurate for calculating addition/removal energies of small organic molecules. Our results indicate that a sufficient dielectric function kinetic energy cutoff may be the limiting step for a wide application of $G_0W_0$ to larger organic systems.

preprint2012arXiv

Random and ordered phases of off-lattice rhombus tiles

We study the covering of the plane by non-overlapping rhombus tiles, a problem well-studied only in the limiting case of dimer coverings of regular lattices. We go beyond this limit by allowing tiles to take any position and orientation on the plane, to be of irregular shape, and to possess different types of attractive interactions. Using extensive numerical simulations we show that at large tile densities there is a phase transition from a fluid of rhombus tiles to a solid packing with broken rotational symmetry. We observe self-assembly of broken-symmetry phases, even at low densities, in the presence of attractive tile-tile interactions. Depending on tile shape and interactions the solid phase can be random, possessing critical orientational fluctuations, or crystalline. Our results suggest strategies for controlling tiling order in experiments involving `molecular rhombi'.

preprint2011arXiv

Electronic energy level alignment at metal-molecule interfaces with a GW approach

Using density functional theory and many-body perturbation theory within a GW approximation, we calculate the electronic structure of a metal-molecule interface consisting of benzene diamine (BDA) adsorbed on Au(111). Through direct comparison with photoemission data, we show that a conventional G$_0$W$_0$ approach can underestimate the energy of the adsorbed molecular resonance relative to the Au Fermi level by up to 0.8 eV. The source of this discrepancy is twofold: a 0.7 eV underestimate of the gas phase ionization energy (IE), and a 0.2 eV overestimate of the Au work function. Refinements to self-energy calculations within the GW framework that account for deviations in both the Au work function and BDA gas-phase IE can result in an interfacial electronic level alignment in quantitative agreement with experiment.

preprint2009arXiv

A note on the metallization of compressed liquid hydrogen

We examine the molecular-atomic transition in liquid hydrogen as it relates to metallization. Pair potentials are obtained from first principles molecular dynamics and compared with potentials derived from quadratic response. The results provide insight into the nature of covalent bonding under extreme conditions. Based on this analysis, we construct a schematic dissociation-metallization phase diagram and suggest experimental approaches that should significantly reduce the pressures necessary for the realization of the elusive metallic phase of hydrogen.

preprint2009arXiv

Structure and phase boundaries of compressed liquid hydrogen

We have mapped the molecular-atomic transition in liquid hydrogen using first principles molecular dynamics. We predict that a molecular phase with short-range orientational order exists at pressures above 100 GPa. The presence of this ordering and the structure emerging near the dissociation transition provide an explanation for the sharpness of the molecular-atomic crossover and the concurrent pressure drop at high pressures. Our findings have non-trivial implications for simulations of hydrogen; previous equation of state data for the molecular liquid may require revision. Arguments for the possibility of a $1^{st}$ order liquid-liquid transition are discussed.