We implemented the well-known Ising model in one dimension as a computer program and simulated its behavior with four algorithms: (i) the seminal Metropolis algorithm; (ii) the microcanonical algorithm described by Creutz in 1983; (iii) a variation on Creutz’s time-reversible algorithm allowing for bonds between spins to change dynamically; and (iv) a combination of the latter two algorithms in a manner reflecting the different timescales on which these two processes occur (“freezing” the bonds in place for part of the simulation). All variations on Creutz’s algorithm were symmetrical in time, and thus reversible. The first three algorithms all favored low-energy states of the spin lattice and generated the Boltzmann energy distribution after reaching thermal equilibrium, as expected, while the last algorithm broke from the Boltzmann distribution while the bonds were “frozen.” The interpretation of this result as a net increase to the system’s total entropy is consistent with the second law of thermodynamics, which leads to the relationship between maximum entropy and the Boltzmann distribution.

The focus of my honors thesis is to find ways to use deep learning in tandem with tools in statistical mechanics to derive new ways to solve problems in biophysics. More specifically, I’ve been interested in finding transition pathways between two known states of a biomolecule. This is because understanding the mechanisms in which proteins fold and ligands bind is crucial to creating new medicines and understanding biological processes. In this thesis, I work with individuals in the Singharoy lab to develop a formulation to utilize reinforcement learning and sampling-based robotics planning to derive low free energy transition pathways between two known states. Our formulation uses Jarzynski’s equality and the stiff-spring approximation to obtain point estimates of energy, and construct an informed path search with atomistic resolution. At the core of this framework, is our first ever attempt we use a policy driven adaptive steered molecular dynamics (SMD) to control our molecular dynamics simulations. We show that both the reinforcement learning (RL) and robotics planning realization of the RL-guided framework can solve for pathways on toy analytical surfaces and alanine dipeptide.

This thesis examines the interpretations derived from the Kac Ring Model, and the adding of a modification to the original model via “kick backs,” which can be interpreted to represent time reversals in the individual Kac rings. The results of this modification are analyzed, and their implications explored. There are three main parts to this thesis. Part 1 is a literature review which explains the working principles of the original Kac ring and explores its numerous applications. Part 2 describes the software and the theoretical & computational methodology used to implement the model and gather data. Part 3 analyzes the data gathered and makes a conclusion about its implications. There is an appendix included which contains some figures from Part 3 in a larger size, as it wasn’t possible to make the figures bigger within the text due to formatting.