Learning from Neural Networks: A Strategy to Identify Systematic Improvements to Activity Coefficient Models

Following widespread adoption of activity coefficient models in routine process design tasks, deep neural networks (DNNs) offer a next-generation approach to the prediction of phase equilibria. Existing work demonstrates that their performance is on-par, if not better, than that of state-of-the-art thermodynamic models,1 albeit single-purpose (e.g., only predicting the infinite dilution coefficient). On the one hand, activity coefficient models can be improved by incorporating more detailed descriptions of reality. On the other hand, identifying sources of systematic error requires careful study, deep understanding of the underlying inaccuracies and potential solutions. Traditional activity coefficient models derived from physics can often be used to calculate a multitude of related properties (e.g., excess enthalpy), and can therefore guide fundamental understanding in ways that many neural models currently cannot.
We posit that a hybrid approach—one where a DNN corrects a traditional activity coefficient model—may offer distinct synergistic advantages. It should improve the overall accuracy through closer representation of experimental data and subsequently that of related properties. Most interesting, however, is the potential to study the model’s corrections to guide the identification of systematic improvements to the activity coefficient model.
In our work, we trained a DNN to not just correct the output of an activity coefficient model, but to adjust its central equations. After all, that is how new relations may be incorporated. This presents several challenges. Firstly, training neural nets requires outputs to be differentiable with respect to the net’s parameters. They need to have a well-defined gradient. Or in other words: to allow a DNN to learn these types of corrections, the activity coefficient model itself, and any (non-linear) solver, needs to be differentiable. That is not a given. Secondly, neural networks are only as accurate as the data they are trained on. Yet, freely available, unlicensed thermodynamic data are rather scarce.
We offer a generalizable approach to learn corrections to undifferentiable activity coefficient models and show that we can markedly improve their performance. We then analyze the resulting corrections to search for new relations. Lastly, we identify potential gaps in available training data that could provide opportunities for further improvements.