Paper detail

Liquified protein vibrations, classification and cross-paradigm de novo image generation using deep neural networks

In recent work we reported the vibrational spectrum of more than 100,000 known protein structures, and a self-consistent sonification method to render the spectrum in the audible range of frequencies (Extreme Mechanics Letters, 2019). Here we present a method to transform these molecular vibrations into materialized vibrations of thin water films using acoustic actuators, leading to complex patterns of surface waves, and using the resulting macroscopic images in further processing using deep convolutional neural networks. Specifically, the patterns of water surface waves for each protein structure is used to build training sets for neural networks, aimed to classify and further process the patterns. Once trained, the neural network model is capable of discerning different proteins solely by analyzing the macroscopic surface wave patterns in the water film. Not only can the method distinguish different types of proteins (e.g. alpha-helix vs hybrids of alpha-helices and beta-sheets), but it is also capable of determining different folding states of the same protein, or the binding events of proteins to ligands. Using the DeepDream algorithm, instances of key features of the deep neural network can be made visible in a range of images, allowing us to explore the inner workings of protein surface wave patter neural networks, as well as the creation of new images by finding and highlighting features of protein molecular spectra in a range of photographic input. The integration of the water-focused realization of cymatics, combined with neural networks and especially generative methods, offer a new direction to realize materiomusical "Inceptionism" as a possible direction in nano-inspired art. The method could have applications for detecting different protein structures, the effect of mutations, or uses in medical imaging and diagnostics, with broad impact in nano-to-macro transitions.