Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.joone.structure
Class Nakayama

    java.lang.Object
      org.joone.structure.Nakayama

All Implemented Interfaces:
  java.io.Serializable, java.util.EventListener, ConvergenceListener, NeuralNetListener, NeuralValidationListener

    public class Nakayama

extends java.lang.Object
implements NeuralNetListener, NeuralValidationListener, ConvergenceListener, java.io.Serializable

This class performs the method of optimizing activation functions as described in:

K.Nakayama and Y.Kimura, "Optimization of activation functions in multilayer neural network applied to pattern classification", Proc. IEEE ICNN'94 Florida, pp.431-436, June 1994.

This techniques probably fails whenever the `NeuralNet.join()` method is called because this optimization technique stops the network to perform the optimization, use a `NeuralNetListener` instead.

Author:
  Boris Jansen
See Also:
  Serialized Form

Constructor Summary
`Nakayama(NeuralNet aNet)` Creates a new instance of Nakayama.

Method Summary
`void` | `addLayer(Layer aLayer)` Adds layers to this optimizer.
`void` | `addLayers(NeuralNet aNeuralNet)` Adds all the hidden layers to this optimizer.
`void` | `cicleTerminated(NeuralNetEvent e)`
`protected void` | `cleanUp()` This method is called after optimization, e.g. to restore the listeners.
`protected void` | `doOptimize()` Optimizes the activation functions of the network.
`void` | `errorChanged(NeuralNetEvent e)`
`protected void` | `evaluateNeurons()` Evaluates neurons, that is, this function calculates information related to the contribution of the activation functions, based on the following three criteria: Information from neurons to its output layers.
`protected int[]` | `findCorrelation(java.util.List aCorrelations, int aLayer, int aNeuron)` Finds a correlation from a given list of correlations.
`protected int` | `findIndex(java.util.List aStatuses, int aLayer, int aNeuron)` Finds the index of a neuron taking into account the deletion of previous neurons.
`protected Layer` | `findInputLayer(Synapse aSynapse)` Finds the input layer of a synapse.
`protected Layer` | `findOutputLayer(Synapse aSynapse)` Finds the output layer of a synapse.
`double` | `getEpsilon()` Gets epsilon, the threshold to decide if a neuron should be deleted or not.
`protected double` | `getGamma(int aLayer1, int aNeuron1, int aLayer2, int aNeuron2)` Gets gammajj' (is equal to gammaj'j).
`protected double[]` | `getMinCorrelation(int aLayer, int aNeuron)` Gets the minimum correlation for a certain neuron j.
`protected double` | `getSumAbsoluteWeights(Layer aLayer, int aNeuron)` Sums up all the absolute values of the output weights of a neuron within a layer.
`protected double[]` | `getSumOutputs(int aLayer, int aNeuron)` Sums up the (normal and absolute) values of the outputs of a neuron over all patterns.
`void` | `netConverged(ConvergenceEvent anEvent, ConvergenceObserver anObserver)` This method is called whenever the network has converged according to some `ConvergenceObserver`
`void` | `netStarted(NeuralNetEvent e)`
`void` | `netStopped(NeuralNetEvent e)`
`void` | `netStoppedError(NeuralNetEvent e, java.lang.String error)`
`void` | `netValidated(NeuralValidationEvent event)`
`boolean` | `optimize()` Optimizes the activation functions of the neural network.
`(package private) void` | `patternFinished()` This method is called after every pattern, so we can retrieve information from the network that is related to the pattern that was just forwarded through the network.
`protected void` | `removeAllListeners()` Removes all the listeners from the neural network (temporarely).
`protected void` | `removeNeuron(int aLayer, int aNeuron)` Removes a neuron.
`protected void` | `restoreAllListeners()` Restore all the listeners to the neural network.
`protected void` | `runValidation()` Runs the network with a validator, this way we are able to collect information related to the different patterns.
`protected void` | `selectNeurons()` Selects neurons to be deleted based on the information calculated by `evaluateNeurons()`.
`void` | `setEpsilon(double anEpsilon)` Sets epsilon, the threshold to decide if a neuron should be deleted or not.
`protected void` | `weightsUpdateCorrelation(java.util.List aStatuses, java.util.List aCorrelations, int aLayer, int aNeuron)` Updates weights before a neuron is removed (because of its similar correlation).
`protected void` | `weightsUpdateVariance(int aLayer, int aNeuronOriginal, int aNeuron)` Updates weights before a neuron is removed (because of its low variance).

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Constructor Detail

### Nakayama

    public Nakayama(NeuralNet aNet)

  Creates a new instance of Nakayama.

  Parameters:
    `aNet` - the network to be optimized.

Method Detail

### addLayer

    public void addLayer(Layer aLayer)

  Adds layers to this optimizer. The layers will be optimized. Layers should be on the same (layer) level, otherwise the optimization does not make sense.

  Parameters:
    `aLayer` - the layer to be added.

### addLayers

    public void addLayers(NeuralNet aNeuralNet)

  Adds all the hidden layers to this optimizer. The layers will be optimized. The neuron network should consist of only one hidden layer, i.e. the hidden layers should be on the same level else this method doesn't make any sense. If the hidden layers are not all on the same level then the layers should be added individually my using , adding only the layers that are on the same hidden level.

  Parameters:
    `aNeuralNet` - the network holding the hidden layers.

### optimize

    public boolean optimize()

  Optimizes the activation functions of the neural network.

  Returns:
    whether the network was optimized or not, i.e. if neurons where deleted or not.

### runValidation

    protected void runValidation()

  Runs the network with a validator, this way we are able to collect information related to the different patterns. Everytime a pattern is forwarded to the network we collect certain info `patternFinished()`.

### doOptimize

    protected void doOptimize()

  Optimizes the activation functions of the network.

### cleanUp

    protected void cleanUp()

  This method is called after optimization, e.g. to restore the listeners.

### selectNeurons

    protected void selectNeurons()

  Selects neurons to be deleted based on the information calculated by `evaluateNeurons()`. It selects neurons to be deleted and performs the deletion.

### weightsUpdateCorrelation

    protected void weightsUpdateCorrelation(java.util.List aStatuses,
                                            java.util.List aCorrelations,
                                            int aLayer,
                                            int aNeuron)

  Updates weights before a neuron is removed (because of its similar correlation).

  Parameters:
    `aStatuses` - the status of the neurons (used to find the correct neuron taking into account any deletions of neurons).
    `aCorrelations` - a list holding all the correlations (neurons to be removed and the correlated neuron (which will take over the weights)).
    `aLayer` - the layer of the neuron to be removed.
    `aNeuron` - the neuron to be removed.

### getGamma

    protected double getGamma(int aLayer1,
                              int aNeuron1,
                              int aLayer2,
                              int aNeuron2)

  Gets gammajj' (is equal to gammaj'j).

  Parameters:
    `aLayer1` - the layer of neuron j.
    `aNeuron1` - the neuron j.
    `aLayer2` - the layer of neuron j'.
    `aNeuron2` - the neuron j'. return gammajj'.

### findIndex

    protected int findIndex(java.util.List aStatuses,
                            int aLayer,
                            int aNeuron)

  Finds the index of a neuron taking into account the deletion of previous neurons.

  Parameters:
    `aStatuses` - the status of neurons.
    `aLayer` - the layer of the neuron.
    `aNeuron` - the index of the neuron, not considering any deletions.
  Returns:
    the index of the neuron taking into account any previous deletions.

### findCorrelation

    protected int[] findCorrelation(java.util.List aCorrelations,
                                    int aLayer,
                                    int aNeuron)

  Finds a correlation from a given list of correlations.

  Parameters:
    `aCorrelations` - a list holding correlations.
    `aLayer` - the layer of the neuron to find the correlation for.
    `aNeuron` - the neuron to find the correlation for.
  Returns:
    the index of the neuron of the correlation with `(aLayer, aNeuron)`, return `null` in case the correlation is not found.

### weightsUpdateVariance

    protected void weightsUpdateVariance(int aLayer,
                                         int aNeuronOriginal,
                                         int aNeuron)

  Updates weights before a neuron is removed (because of its low variance).

  Parameters:
    `aLayer` - the index of the layer of the neuron.
    `aNeuronOriginal` - the index of the neuron to be removed (NOT taking into account previous deletions).
    `aNeuron` - the index of the neuron to be removed (taking into account previous deletions).

### removeNeuron

    protected void removeNeuron(int aLayer,
                                int aNeuron)

  Removes a neuron.

  Parameters:
    `aLayer` - the index of the layer in which we should remove the neuron.
    `aNeuron` - the index of the neuron to be removed (taking into account previous deletions).

### findInputLayer

    protected Layer findInputLayer(Synapse aSynapse)

  Finds the input layer of a synapse.

  Parameters:
    `aSynapse` - the synapse to find the input layer for.

### findOutputLayer

    protected Layer findOutputLayer(Synapse aSynapse)

  Finds the output layer of a synapse.

  Parameters:
    `aSynapse` - the synapse to find the output layer for.

### evaluateNeurons

    protected void evaluateNeurons()

  Evaluates neurons, that is, this function calculates information related to the contribution of the activation functions, based on the following three criteria:
  - Information from neurons to its output layers.
  - Variance of the output of the neurons.
  - Correlation between outputs of neurons.
  This information will be used in a next stage to select neurons to delete.

### getMinCorrelation

    protected double[] getMinCorrelation(int aLayer,
                                         int aNeuron)

  Gets the minimum correlation for a certain neuron j. A correlation between neuron j and j' is defined as Rjj' = 1 - |Gammajj'|. Rjmin (which this function calculates is defined as Rjmin = min-j{Rjj'}. It also returns the index of the neuron j that is the argument to this function as well as j' that is the minimum.

  Parameters:
    `anLayer` - the index of the layer of the neuron `aNeuron`.
    `aNeuron` - the neuron within the layer (j).
  Returns:
    the minimum correlation Rjmin, together with the index of the neuron as an argument and the neuron of the minimum (layer and neuron). The lower index of the two neurons is at position 1, 2 and the higher index is at position 3, 4. The minimum itself is at position 0. Finally at position 5 we will indicate if the argument was the lower index neuron (<0, so it is now at position 1, 2) or if the argument was the higher index neuron (>0, so now it is at position 3, 4).

### getSumOutputs

    protected double[] getSumOutputs(int aLayer,
                                     int aNeuron)

  Sums up the (normal and absolute) values of the outputs of a neuron over all patterns.

  Parameters:
    `aLayer` - an index of the layer to retrieve the outputs of that layer.
    `aNeuron` - the neuron in the layer.
  Returns:
    the sum of (index 0: normal, index 1: absolute) outputs of neuron `aNeuron`.

### getSumAbsoluteWeights

    protected double getSumAbsoluteWeights(Layer aLayer,
                                           int aNeuron)

  Sums up all the absolute values of the output weights of a neuron within a layer.

  Parameters:
    `aLayer` - the layer holding neuron `aNeuron`.
    `aNeuron` - the neuron in the layer.
  Returns:
    the sum of absolute values of the output weights of neuron `aNeuron` within layer `aLayer`.

### cicleTerminated

    public void cicleTerminated(NeuralNetEvent e)

  Specified by:
    `cicleTerminated` in interface `NeuralNetListener`

### errorChanged

    public void errorChanged(NeuralNetEvent e)

  Specified by:
    `errorChanged` in interface `NeuralNetListener`

### netStarted

    public void netStarted(NeuralNetEvent e)

  Specified by:
    `netStarted` in interface `NeuralNetListener`

### netStopped

    public void netStopped(NeuralNetEvent e)

  Specified by:
    `netStopped` in interface `NeuralNetListener`

### netStoppedError

    public void netStoppedError(NeuralNetEvent e,
                                java.lang.String error)

  Specified by:
    `netStoppedError` in interface `NeuralNetListener`

### netValidated

    public void netValidated(NeuralValidationEvent event)

  Specified by:
    `netValidated` in interface `NeuralValidationListener`

### removeAllListeners

    protected void removeAllListeners()

  Removes all the listeners from the neural network (temporarely). They will be added again after optimization and the network is restarted.

### restoreAllListeners

    protected void restoreAllListeners()

  Restore all the listeners to the neural network.

### patternFinished

    void patternFinished()

  This method is called after every pattern, so we can retrieve information from the network that is related to the pattern that was just forwarded through the network.

### getEpsilon

    public double getEpsilon()

  Gets epsilon, the threshold to decide if a neuron should be deleted or not.

  Returns:
    the threshold epsilon.

### setEpsilon

    public void setEpsilon(double anEpsilon)

  Sets epsilon, the threshold to decide if a neuron should be deleted or not.

  Parameters:
    `anEpsilon` - the new epsilon.

### netConverged

    public void netConverged(ConvergenceEvent anEvent,
                             ConvergenceObserver anObserver)

  Description copied from interface: `ConvergenceListener`
  This method is called whenever the network has converged according to some `ConvergenceObserver`

  Specified by:
    `netConverged` in interface `ConvergenceListener`
  Parameters:
    `anEvent` - the event that is generated.
    `anObserver` - the observer that generated the event.

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Submit Feedback to pmarrone@users.sourceforge.net