The applet requires Java 5 or higher. Java must be enabled in your browser settings. Mac users must have Mac OS X 10.4 or higher. Windows and Linux users may obtain the latest Java from Oracle's Java site.

Welcome to this simulation of a pattern memorization and recognition using a neural network. Here's a quick guide on how to use it:

  1. Press the "Setup inputs" button to initialize the neural network with random weights and cleared inputs. The default is to have 3 input patterns, but you can adjust that by moving the "inputs" slider anywhere from 1 to 5 input patterns.
  2. Press the "Draw" button to draw any patterns that you want within each of the initially black picture boxes. Draw your pattterns with the mouse, pressing the mouse's button to leave behind white ink. If you make a mistake, deselect the "Draw" button, and select the "Erase" button instead.
  3. Press "Memorize patterns" to have the neural network learn based on all of the input patterns. During learning, the patterns are rapidly and randomly presented to the network, and it uses a modified version of Hebbian learning to adjust the weights between nodes. There is a neural network node for every image pixel, and there is a weight (a connection strength) that connects every node to every other node. A node will have a value of +1 if its pixel is white in a picture and -1 if the pixel is black. After learning, nodes/pixels that tend to have the same state (either both are white or both are black within a picture) across all of input pictures will tend have positive connections/weights between them. Nodes that tend to be in opposite states will have negative weights. If one pixel doesn't predict the state another pixel at all, then the two pixels will have a connection strength of 0 between them.
  4. The "Memory" window shows the entire state of learned weights for the network. If there are N pixels in an image, then there are N^2 weights because every node is connected to every other node. We can't show all of these weights at the same time, so we show all of the weights to the node that is colored red. To change which node is colored red, press the "Probe Memory" button and then select the node in the "Memory" window for which you would like to see its connections to other nodes. Deselect "Probe Memory" when you are done probing memory. The weights are shown by their color: black = a weight of -1 between the pixel's node and the node in red, white = a weight of +1 between the nodes, intermediate brightness = a weight somewhere between -1 and +1 with brighter colors showing larger weights.
  5. To test the neural network's pattern recognition and memory retrieval capabilities, draw a pattern in the "Input" box, or copy one of the pictures you drew before. To copy a picture, first press the "Copy" button, and then click on the picture you want to copy. It will show up in the "From box" window. Then, click on the input box and it will appear in the "To box" window. The "output" window will originally look like the "input" window, but when you press the "process input" button, then the activations of the nodes representing pixels will be influenced by the memory and the other pixels. Specifically, Node P's activation will be changed by the sum of the products of the other nodes connected to P multiplied by the connection weight between the nodes and P. For example, if Node C has two other nodes connected to it, Nodes A and B, and the connection strength between A and C is +0.2, the connection strength between B and C is -0.1, the activation of A is +0.4 and the activation of B is -0.3, then the influence coming into C will be (0.2*0.4)+(-0.1*-0.3)= +0.83. C will increase its activation according to this influence, and so even if C was originally negatively activated, it will tend to become positively activated because of the influence of memory.
  6. You can add random noise to the input by pressing the "Degrade input" button. You can also draw or erase elements of the input by selecting the "draw" or "erase" button and using your mouse and button. Noise can be added to the memory itself by pressing the "degrade" memory button, and you can also manually draw or erase elements of memory.
  7. Things to explore: draw pictures of "A," "B", and "C." Memorize then patterns. Then, copy "B" to the input window, add noise to it, and erase a piece of it. Press "process input." Is the originally memorized "B" picture reconstructed? How much can you distort the "B" before the network fails to reproduce it? How much can you destroy the actual memory before the network fails to reproduce it? Probe the memory and see if you can predict what connection strength will exist between two nodes based upon the particular input patterns you drew. What happens when you draw an input picture that is halfway between two memorize pictures? What happens when you only draw a couple of pixels in the input pattern and try to process it? What happens if you begin with all input pixels turned ON, and then you ERASE some of the pixels associated with a memorized pattern - drawing black on white instead of white on black?

powered by NetLogo

view/download model file: hopfield.nlogo

Developed by Robert Goldstone

Visit the Percepts and Concepts Laboratory