botiverse.models.NN package#
Submodules#
botiverse.models.NN.NN module#
This module implements and provides an interface for an arbitrary neural network architecture.
- class botiverse.models.NN.NN.NeuralNet(structure, activation='sigmoid', optimizer='ADAM')[source]#
Bases:
objectDefines the hypothesis set and learning algorithm for a neural network.
Initialize the hyperparameters, weights and biases for the network.
- Parameters:
structure (list) – A list of integers representing the number of neurons in each layer. For example, [2, 3, 1] is a network with 2 neurons in the input layer, 3 in the hidden layer, and 1 in the output layer.
activation (str) – The activation function to use. Can be ‘sigmoid’ or ‘relu’.
optimizer (str) – The optimizer to use. Can be ‘ADAM’ or ‘SGD’.
- ADAM(x_batch, y_batch, λ, α=0.01, beta1=0.9, beta2=0.999, epsilon=1e-08)#
Given a minibatch (a list of tuples (xₛ, yₛ )) this will update Bₙ and Wₙ by applying Adam optimizer with L2 regularization.
- Parameters:
x_batch (numpy.ndarray or list) – A list/numpy array of input vectors of shape (d, 1).
y_batch (numpy.ndarray or list) – A list/numpy array of output vectors of shape (K, 1).
λ (float) – The learning rate.
α (float) – The regularization parameter.
beta1 (float) – The exponential decay rate for the first moment estimates.
beta2 (float) – The exponential decay rate for the second-moment estimates.
epsilon (float) – A small constant for numerical stability.
- SGD(x_batch, y_batch, λ, α=0.01)#
Given a minibatch (a list/numpy array of tuples (xₛ, yₛ )) this will update Bₙ and Wₙ by applying SGD with L2 regularization.
- Parameters:
x_batch (numpy.ndarray or list) – A list/numpy array of input vectors of shape (d, 1).
y_batch (numpy.ndarray or list) – A list/numpy array of output vectors of shape (K, 1).
λ (float) – The learning rate.
α (float) – The regularization parameter.
- backprop(xs, ys)#
Compute the loss gradients მJⳆმBₙₛ, მJⳆმWₙₛ given an observation (xₛ , yₛ ) where xₛ and yₛ are column vectors.
- Parameters:
xₛ (numpy.ndarray) – The input vector of shape (d, 1).
yₛ (numpy.ndarray) – The output vector of shape (K, 1).
- Returns:
A tuple of lists (მJⳆმBₙₛ, მJⳆმWₙₛ) where მJⳆმBₙₛ is a list of the gradients of the loss function with respect to the biases and მJⳆმWₙₛ is a list of the gradients of the loss function with respect to the weights.
- Return type:
tuple of lists
- evaluate(X, y, loss=False)#
Compare the one-hot vector y with the networks output yᴺ and calculate the accuracy.
- Parameters:
X (numpy.ndarray) – The test data arranged as a float numpy array of shape (N, d)
y (numpy.ndarray) – The test labels arranged as a numpy array of shape (N,) where each element is an integer between 0 and k-1
- feedforward(x)#
- fit(X, y, batch_size=32, epochs=200, λ=30, α=0.01, optimizer='ADAM', val_split=0.0, patience=50, eval_train=False)#
For each epoch, go over each minibatch and perform a gradient descent update accordingly and evaluate the model on the training and validation sets if needed.
- Parameters:
X (numpy.ndarray) – The training data arranged as a float numpy array of shape (N, d)
y (numpy.ndarray) – The training labels arranged as a numpy array of shape (N,) where each element is an integer between 0 and k-1
batch_size (int) – The size of the minibatches to use.
epochs (int) – The number of epochs to run.
λ (float) – The learning rate.
α (float) – The regularization parameter.
optimizer (str) – The optimizer to use. Can be ‘ADAM’ or ‘SGD’.
val_split (float) – The ratio of the validation set to the training set. If given, per-epoch validation will be performed.
eval_train (bool) – Whether to evaluate the model on the training set.
- load()#
Load the model from the given path.
- Parameters:
path (str) – The path to load the model from.
- predict(X)#
Predict the class of each example in X.
- Parameters:
X (numpy.ndarray) – The test data arranged as a float numpy array of shape (N, d)
- save(path)#
Save the model to the given path.
- Parameters:
path (str) – The path to save the model to.
- botiverse.models.NN.NN.NNClass(func)#
- botiverse.models.NN.NN.backprop(self, xs, ys)[source]#
Compute the loss gradients მJⳆმBₙₛ, მJⳆმWₙₛ given an observation (xₛ , yₛ ) where xₛ and yₛ are column vectors.
- Parameters:
xₛ (numpy.ndarray) – The input vector of shape (d, 1).
yₛ (numpy.ndarray) – The output vector of shape (K, 1).
- Returns:
A tuple of lists (მJⳆმBₙₛ, მJⳆმWₙₛ) where მJⳆმBₙₛ is a list of the gradients of the loss function with respect to the biases and მJⳆმWₙₛ is a list of the gradients of the loss function with respect to the weights.
- Return type:
tuple of lists
- botiverse.models.NN.NN.SGD(self, x_batch, y_batch, λ, α=0.01)[source]#
Given a minibatch (a list/numpy array of tuples (xₛ, yₛ )) this will update Bₙ and Wₙ by applying SGD with L2 regularization.
- Parameters:
x_batch (numpy.ndarray or list) – A list/numpy array of input vectors of shape (d, 1).
y_batch (numpy.ndarray or list) – A list/numpy array of output vectors of shape (K, 1).
λ (float) – The learning rate.
α (float) – The regularization parameter.
- botiverse.models.NN.NN.ADAM(self, x_batch, y_batch, λ, α=0.01, beta1=0.9, beta2=0.999, epsilon=1e-08)[source]#
Given a minibatch (a list of tuples (xₛ, yₛ )) this will update Bₙ and Wₙ by applying Adam optimizer with L2 regularization.
- Parameters:
x_batch (numpy.ndarray or list) – A list/numpy array of input vectors of shape (d, 1).
y_batch (numpy.ndarray or list) – A list/numpy array of output vectors of shape (K, 1).
λ (float) – The learning rate.
α (float) – The regularization parameter.
beta1 (float) – The exponential decay rate for the first moment estimates.
beta2 (float) – The exponential decay rate for the second-moment estimates.
epsilon (float) – A small constant for numerical stability.
- botiverse.models.NN.NN.fit(self, X, y, batch_size=32, epochs=200, λ=30, α=0.01, optimizer='ADAM', val_split=0.0, patience=50, eval_train=False)[source]#
For each epoch, go over each minibatch and perform a gradient descent update accordingly and evaluate the model on the training and validation sets if needed.
- Parameters:
X (numpy.ndarray) – The training data arranged as a float numpy array of shape (N, d)
y (numpy.ndarray) – The training labels arranged as a numpy array of shape (N,) where each element is an integer between 0 and k-1
batch_size (int) – The size of the minibatches to use.
epochs (int) – The number of epochs to run.
λ (float) – The learning rate.
α (float) – The regularization parameter.
optimizer (str) – The optimizer to use. Can be ‘ADAM’ or ‘SGD’.
val_split (float) – The ratio of the validation set to the training set. If given, per-epoch validation will be performed.
eval_train (bool) – Whether to evaluate the model on the training set.
- botiverse.models.NN.NN.predict(self, X)[source]#
Predict the class of each example in X.
- Parameters:
X (numpy.ndarray) – The test data arranged as a float numpy array of shape (N, d)
- botiverse.models.NN.NN.evaluate(self, X, y, loss=False)[source]#
Compare the one-hot vector y with the networks output yᴺ and calculate the accuracy.
- Parameters:
X (numpy.ndarray) – The test data arranged as a float numpy array of shape (N, d)
y (numpy.ndarray) – The test labels arranged as a numpy array of shape (N,) where each element is an integer between 0 and k-1
- botiverse.models.NN.NN.save(self, path)[source]#
Save the model to the given path.
- Parameters:
path (str) – The path to save the model to.
- botiverse.models.NN.NN.load()#
staticmethod(function) -> method
Convert a function to be a static method.
A static method does not receive an implicit first argument. To declare a static method, use this idiom:
- class C:
@staticmethod def f(arg1, arg2, …):
…
It can be called either on the class (e.g. C.f()) or on an instance (e.g. C().f()). Both the class and the instance are ignored, and neither is passed implicitly as the first argument to the method.
Static methods in Python are similar to those found in Java or C++. For a more advanced concept, see the classmethod builtin.