PNNL

Abstract

Sustainable liquid fuel hydrocarbons higher energy densities compared to other technologies and thus play a role in the decarbonization of the transportation sector, especially in long-range applications such as aviation, marine and rail. Through thermochemical pathways to produce liquid from biomass such as pyrolysis and hydrothermal liquefaction, production of hydrocarbons entails hydrotreating reactions where heteroatoms, especially oxygen, nitrogen, and sulfur, are catalytically removed in the presence of hydrogen. This study investigated the utilization of the molecular transformer (MT) model to predict hydrotreating (HT) reactions given starting biomass-derived compounds. Three different approaches to model implementation were investigated: training the model from scratch on all reactions, fine-tuning a pre-trained MT model on a limited HT reaction dataset, and modifying the MT model to incorporate reaction conditions as new features. The models were evaluated using Simplified molecular-input line-entry system (SMILES) and SELF-referencing embedded string (SELFIES) representations. A combination of manually created dataset consisting of HT reactions mined from peer-reviewed literature and a generic chemical reactions dataset from the U.S. Patent and Trademark Office (USPTO) were used to assess the various MT models’ feasibility for HT reactions. The results demonstrate that the models can learn and utilize relevant chemical features, catalysts, and reaction conditions to predict the outcome of HT reactions. The cumulative accuracy of predicting elementary reactions from a test set consisting of HT-relevant compounds using the modified MT models reached more than 70%, a very significant improvement from the initial 20% with the vanilla MT model. This study represents a significant step towards utilizing deep learning to predict HT reactions and provides valuable insights for future advancements in this field.