Everything not in “Can Neural Networks Think Like Geomorphologists”
I recently finished up a paper “Can Neural Networks Think Like Geomorphologists” which should be being published shortly (is it bad form to say where?). The preprint is linked above, and I’ll have a more casual write up on the webpage soon. But there’s a lot that didn’t make it into the paper. Little dead ends that didn’t go anywhere, things that were helpful to guide my thinking but didn’t have a place in the manuscript, and some stubs that have potential but I didn’t fully develop. They couldn’t go in the paper without making it a mess, but I thought it would be good preserve them somewhere, hence this blog post. I’m not going to go over the project context, so take a look at the manuscript or the project page.
Péclet Numbers
This paper is about understanding an advective diffusive system by the ratio of diffusive transport (D) to advective transport (K). This is often (in both the geomorphology literature and beyond) with a Péclet number - a non dimensionalized ratio of advective to diffusive transport. However, in my paper I use \(D/K\), which has not been non dimensionalized, and is the inverse ratio. It didn’t start out this way. The very first neural networks I trained were for \(K/D\) with the understanding I would nondimensionalize it later. I was eager to get the project off the ground, and wanted to put off choosing the characteristic scales necessary for nondimensionalization. Eventually I reviewed a few approaches for determining length scales, chose one, and trained a bunch of neural networks to infer Péclet instead of raw \(K/D\). The network performed just as well (which makes sense since I used uniform characteristic scales for all my landscapes). Then, after some urging by others, I tried the neural network on some real landscapes. I realized that my choices of scale were perhaps sensible for my modeled input, but not for these real landscapes. The real data was reported as \(D/K\) values Perron et al. 2009 so I re-trained my networks to infer \(D/K\). The real landscapes didn’t work, so they were discarded. When preparing the work for publication, we decided that nondimensionalizing the parameters didn’t really help us understand anything, and required decisions about scales that were hard to make, so we stuck with \(D/K\). In this blog post I’m bringing back figures from earlier stages of this project that were generated when we were using \(K/D\) and Péclet, so just understand that there was some drift in presentation, but it all means basically the same thing.
Interpretability Techniques
The paper is really an interpretability paper. I built this neural network to see how we could take it apart as geomorphologists, and if it might be possible to have neural networks teach us geomorphology, not just solve problems (by which I mean classify or predict something with really high performance). This was actually the source of a bit of frustration when I would present this work; people would inevitability ask something like “oh, can we use this to get \(D/K\) values from real landscapes?”, and you couldn’t because the models I used to train that were too uniform, in part because I though everyone would scoff at a neural network trained on models being used to say anything about real data. Now that I realize the field isn’t hostile to that, it’s something we’re working on to some extent, but we’ll see where that goes. Anyway the point of the study was not “lets build a \(D/K\) predictor”, it was “is it crazy to try to use CNN’s to find new geomorphically meaningful spatial patterns?”.
I was really excited about convolutional neural networks because of all the interpretability techniques that the computer vision field has come up with, and so my idea was to just try them on my network.
Activation Maximization
I started with activation maximization, which briefly is a way of creating an input that maximizes the output of a specific neuron in the neural network. If the neuron outputs a value between 0 and 1, the activation maximization image gets that neuron very close to 1. Our network has convolutions, which have 2D outputs, so think of the output we’re targeting as close to 1 everywhere. I generated these for the neurons in my network (you can see it for every neuron here), but they mostly look like this:
This is pretty noisy but maybe you can convince yourself that there is some structure, some sort of evenly spaced wave type thing. Well that’s something since we know that \(D/K\) (what this network is inferring) is related to valley spacing. Could this be proof of a valley spacing relationship? This was promising but not really enough, there wasn’t any obvious structure, like systematically different orientations or spacing or anything like that. To try to test that the network was sensitive to valley spacing, I ended up constructing a series of sine inputs, and that made it into the paper.
GradCAM
GradCAM is another interpretation technique that highlights areas of the input that are important for the output. I constructed grad cam images for the network as a whole and each convolutional layer, for an example low \(D/K\) (“High Peclet” in the figure) and high \(D/K\) (“Low Peclet”) in the figure.
It’s not nothing, but I didn’t see anything on here that I could really build towards. The landscapes are symmetrical so (the third layer) learns to focus on the right side? Early layers look at finer scale valley features? Again it’s not nothing, but between this and the activation maximization images, it was shaping up to be a very handwavy paper, and I didn’t have things I could tie to the geomorphology like I wanted to. An important thing to note, is that these methods, like a lot of things in computer vision, were developed for networks that are classifying their inputs. I think you can go a lot further when you can tie these images back to specific classes, and maybe when your inputs are a bit more heterogeneous.
Ablation and weight visualization
One thing that was looking really exciting for a while was ablating neurons, and visually inspecting the weights of the most important neurons. I found some important neurons that looked like they could be doing ridge and face detection.
To test this, I created a version of the neural network where I fixed the first set of convolutions to be explicitly focused on detecting ridges and faces. The resulting network did OK, but I suspect that it’s spite of the ridge and face convolutions, not because of them.
Fourier Decomposition
Inspired by some other work on landscapes Perron et al. 2008 I did some Fourier decompositions of the landscape as it got transformed by the neural network. This yielded some cool plots, but not much to work with it. I’m not convinced that there isn’t something there.
Spatial Scale Ablation Experiments
I created a neural network that had convolutions with different windows running in parallel, and then ablated based on window size to see how that impacted network performance. I was hoping for something like “ablating this window size means that the network loses skill in this range of inputs but not this other range”. Instead it just would totally not perform. I think this experiment could go somewhere if I designed it a little more thoughtfully.