Computational power over the last decade has increased rapidly, partially due to the popularization of the graphics processing unit (GPU). Due to this, Large Eddy Simulation (LES) has begun to grow in prominence in fluid simulations of engineering interest. Subgrid scale (SGS) modeling plays a crucial role in the accuracy of LES. Therefore, improvements in SGS modeling are of interest to more accurately simulate engineering fluid flows. Due to the availability of data and the maturation of machine learning, neural network driven SGS modeling is an active research area. However, several remaining questions can still be answered. In the age of increasing neural network complexity, what inductive biases are needed for neural network SGS modeling? Do these inductive biases (and increased neural network complexity) help create more accurate closures?