MachineLearning::NeuralNetwork – create, train, test, resize, store, and run a neural network
use MachineLearning::NeuralNetwork; my $nn = MachineLearning::NeuralNetwork->new({ "Name" => "Sample Network", "Description" => "Sample network", "HiddenLayerSize" => 5, "InputFieldNames" => ["Input1", "Input2"], "OutputFieldNames" => ["Output"]}); my $nn = MachineLearning::NeuralNetwork->open( "~/neural_networks/sample.nn"); my $success = $nn->get_success(); my $message = $nn->get_message(); my $network_name = $nn->get_name(); my $network_description = $nn->get_description(); my $ran_ok = $nn->run($data_file_path); my $trained_ok = $nn->train({ "TrainingDataFilePath" => $training_data_file_path, "CyclesPerTemperature" => $cycles_per_temperature, "MinimumOnesPercentage" => $minimum_ones_percentage}); my $test_results = $nn->test($validation_data_file_path); my $new_hidden_layer_size = $nn->grow({ "TrainingDataFilePath" => $training_data_file_path, "ValidationDataFilePath" => $validation_data_file_path, "CyclesPerTemperature" => $cycles_per_temperature, "MinimumOnesPercentage" => $minimum_ones_percentage}); my $new_hidden_layer_size = $nn->prune({ "TrainingDataFilePath" => $training_data_file_path, "ValidationDataFilePath" => $validation_data_file_path, "CyclesPerTemperature" => $cycles_per_temperature, "MinimumOnesPercentage" => $minimum_ones_percentage}); my $saved_ok = $nn->save("~/neural_networks/perfected_network.nn");
This module defines a package for creating a MachineLearning::NeuralNetwork object.
This module uses the MachineLearning::SimulatedAnnealing module to optimize the network's weights and thresholds during training.
A neural network as implemented by the MachineLearning::NeuralNetwork module favors quality over quantity. That is, it is optimized to find the highest quality predictions for a single result value without particular regard to how many data records (or instances) get screened out in the process. This is highly useful for many applications. For example, out of many potential financial investments, you might want to identify a small group that have unusually good prospects for success.
The result values supported by the MachineLearning::NeuralNetwork
module are 1
and 0
, and the accuracy of
the results is important only for the 1
values. To
ensure that the neural network's output layer generates a
satisfactory minimum quantity of ones, the methods for training,
growing, and pruning the neural network take an optional
MinimumOnesPercentage argument.
To use this module, you must have both the MachineLearning module and the MachineLearning::SimulatedAnnealing module installed.
This is the constructor.
In addition to the class-name argument, which is passed
automatically when you use the
MachineLearning::NeuralNetwork->new()
syntax,
the constructor takes a reference to a hash containing the
following keys:
Name Description HiddenLayerSize InputFieldNames OutputFieldNames
The Name and Description must be non-empty strings. The HiddenLayerSize must be a positive integer specifying the number of neurons in the neural network's hidden layer. The value associated with the InputFieldNames key must be a reference to an array of input field names. The value associated with the OutputFieldNames key must be a reference to an array containing exactly one output field name.
All field names (for input and output fields combined) must be unique. Field names must not contain commas or line-break characters. There must be at least two input field names and exactly one output field name.
The constructor returns a reference to a MachineLearning::NeuralNetwork object, which is implemented internally as a hash. All functionality is exposed through methods.
If the constructor receives a valid hash reference providing
all required information, the get_success()
instance method returns true (1
) and the
get_message()
instance method returns an empty
string; otherwise, get_success()
returns false
(0
) and get_message()
returns a
string containing an error message.
This returns true (1
) if the neural network
object was initialized successfully; otherwise, it returns
false (0
).
When get_success()
returns true
(1
), this returns an empty string. When
get_success()
returns false (0
),
get_message()
returns an error message.
Returns the name of the neural network, or an empty string if the neural network was never successfully initialized.
Returns the description of the neural network, or an empty string if the neural network was never successfully initialized.
This method runs the neural network on the specified data.
The specified data file must be in CSV format with a header row. The header row must contain the names of the input neurons in the correct order followed by the name of the output neuron. NOTE: There can be more than one output field; however, the neural network uses and preserves only the one for which the column heading matches the output-field name associated with the neural network.
Each record in the data must contain the correct number of input values as well as a blank or replaceable output value in the appropriate output field. IMPORTANT: The method removes all other output fields from the file.
The method adds the neural network–generated output value to each record, overwriting the output value that is already there, if any. NOTE: Input and output values must not contain commas or line-break characters.
If everything goes OK, the method returns true
(1
); otherwise, the method returns false
(0
). If the neural network was in a valid state
previously but something went wrong during execution of the
run()
method, the method sets the
_success
field (returned by the
get_success()
method) to false (0
)
and places an error message in the _message
field
(returned by the get_message()
method).
$args
is a reference to a hash containing the
following keys:
TrainingDataFilePath CyclesPerTemperature MinimumOnesPercentage
This method trains the neural network using the specified
data and the specified number of cycles per temperature. The
value specifying the minimum percentage of ones required for
the output node to avoid an automatic cost assessment of
100
is optional and, if missing or invalid, is
assumed to be 0
. Even if the minimum ones
percentage is 0
, however, there must always be at
least one occurrence of a 1
result for the output
neuron to avoid an automatic cost assessment of
100
.
The training data must be in CSV format with a header row. The header row must contain the names of the input neurons in the correct order followed by the name of the output neuron. NOTE: There can be more than one output field; however, the neural network uses only the one for which the column heading matches the output field name associated with the neural network.
Each record in the data must contain the correct number of input values as well as an output value in the appropriate output field. The output value supplied with the training data is typically the expected, ideal, or real-life result for the supplied input values. NOTE: Input and output values must not contain commas or line-break characters.
CyclesPerTemperature is a positive integer (for example,
1_000
) specifying the number of randomization
cycles performed at each temperature level during the
simulated annealing process. (For more information, see the
MachineLearning::SimulatedAnnealing module).
During training, the network minimizes cost using
simulated annealing to optimize the weights and thresholds.
Cost is a number that represents how much error results
when a particular set of weights and thresholds is applied to
the training data. The cost is the percentage of the time that
the 1
values in the output generated by the
network do not match the corresponding values provided by the
training data.
If specified, the MinimumOnesPercentage value must be a
positive number less than 100 that represents a percentage.
The cost calculated for the output neuron's values over a
data set will be set to 100
automatically if the
percentage of ones is less than the specified minimum (or if
there are no ones at all).
If everything goes OK, the method returns true
(1
); otherwise, the method returns false
(0
). If the neural network was in a valid state
previously but something went wrong during execution of the
train()
method, the method sets the
_success
field (returned by the
get_success()
method) to false (0
)
and places an error message in the _message
field
(returned by the get_message()
method).
This method tests the network using the supplied validation data, which must be in the same format as the training data.
The method returns a string containing the test results in the form of a formatted report, which gives the cost for the output neuron.
Cost is a number that represents how much error
results when the neural network is applied to the test data.
The cost is the percentage of the time that the 1
values in the output generated by the network do not match the
corresponding values provided by the test data.
During training, the network minimizes cost using simulated annealing to optimize the weights and thresholds. During testing, however, there are no adjustments to the weights and thresholds; the results are simply calculated and reported.
TIP: Testing reveals how well the network generalizes to out-of-sample data. Therefore, make sure that the validation data does not overlap with the training data. To compare the test results with the results of applying the network to the data on which it was trained, you can run a test using the training data. The cost is typically higher for the test data, so the important question is whether that cost is sufficiently low for the network to be useful.
If something goes wrong during execution of the method, the
method returns an empty string and, if the
_success
field (returned by the
get_success()
method) is currently set to true
(1
), sets that field to false (0
)
and places an error message in the _message
field
(returned by the get_message()
method).
$args
is a reference to a hash containing the
following keys:
TrainingDataFilePath ValidationDataFilePath CyclesPerTemperature MinimumOnesPercentage
The MinimumOnesPercentage key is optional.
This method grows the neural network by performing training and testing with a progressively increasing number of hidden neurons. The size of the hidden layer starts at five and then progresses upward through the Fibonacci series to 144 (that is, the sizes used are 5, 8, 13, 21, 34, 55, 89, and 144). Once the neural network has been trained and tested with a hidden layer size of 144, the method chooses the size with the best result (the lowest cost) based on post-training validation, retrains the network with that number of hidden neurons, and then returns that number. NOTE: In the case of a tie, the method favors the smaller number of hidden neurons.
If something goes wrong during execution of the method, the
method returns 0
and, if the
_success
field (returned by the
get_success()
method) is currently set to true
(1
), sets that field to false (0
)
and places an error message in the _message
field
(returned by the get_message()
method).
$args
is a reference to a hash containing the
following keys:
TrainingDataFilePath ValidationDataFilePath CyclesPerTemperature MinimumOnesPercentage
The MinimumOnesPercentage key is optional.
This method prunes the neural network by performing training followed by testing with a progressively decreasing number of hidden neurons. The size of the hidden layer decreases by one for each cycle of training and testing. Once all sizes have been tried from the initial size down to the closest lower number that is in the Fibonacci series, the method chooses the size with the best result (the lowest cost), retrains the network with that number of hidden neurons, and returns that number. NOTE: In the case of a tie, the method favors the smaller number of hidden neurons.
If something goes wrong during execution of the method, the
method returns 0
and, if the
_success
field (returned by the
get_success()
method) is currently set to true
(1
), sets that field to false (0
)
and places an error message in the _message
field
(returned by the get_message()
method).
This method saves (serializes) the neural network object to a file. A neural network must be already trained before you can save it.
If the serialization and file-writing operations succeed,
this method returns true (1
); otherwise, the
method sets the _success
field to false
(0
), places an error message in the
_message
field, and returns false
(0
).
This method returns a new MachineLearning::NeuralNetwork
object created by restoring such an object from the specified
file. If the file-reading and deserialization operations
succeed, the resulting object's get_success()
method returns true (1
) and the
get_message()
method returns an empty string.
Otherwise, the open()
method creates and returns
a new object that has a false value in the
_success
field and an error message in the
_message
field.
All input values must be decimal numbers in the range
-1
to 1
, inclusive.
Internally, the neural network uses weight values in the range
-1
to 1
(inclusive) and thresholds in
the range 0.00000001
to n, where n is
the number of neurons in the preceding layer. Both the hidden and
output layers have thresholds, and the output value is determined
by whether the threshold for the output node is reached
(1
if yes, 0
if no).
All output values provided by training data or validation data
must be either 0
or 1
.
This module uses a feed-forward neural network architecture with
one hidden layer. The number of hidden nodes is variable, and the
recommended approach is to try various numbers in ascending order
(for example, by using the grow()
method). Then,
starting with the number that produced the best results based on
post-training validation, prune the neural network using the
prune()
method.
TIP: You can grow and then prune a neural network several times using different data sets in order to gain more insight into the optimal size for the hidden layer. You can also switch the training and validation sets to get twice as many train-and-test cycles from your data. When using these approaches, consider reserving sufficient data for a final test; data that is not part of any of the data sets that you are using for training and validation during the development phase of the neural network. If the final test is not satisfactory, you might have to reconsider the types of inputs that you are using for the neural network, gather sufficient additional data for a new final test, and then develop the neural network again using a different input framework.
Benjamin Fitch, <blernflerkl@yahoo.com>
Copyright 2009 by Benjamin Fitch
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.