Algorithms_in_C++  1.0.0
Set of algorithms implemented in C++.
adaline_learning.cpp File Reference

Adaptive Linear Neuron (ADALINE) implementation More...

#include <array>
#include <cassert>
#include <climits>
#include <cmath>
#include <cstdlib>
#include <ctime>
#include <iostream>
#include <numeric>
#include <vector>
Include dependency graph for adaline_learning.cpp:

Classes

class  machine_learning::adaline
 

Namespaces

 machine_learning
 Machine learning algorithms.
 

Functions

void test1 (double eta=0.01)
 
void test2 (double eta=0.01)
 
void test3 (double eta=0.01)
 
int main (int argc, char **argv)
 

Variables

constexpr int MAX_ITER = 500
 

Detailed Description

Adaptive Linear Neuron (ADALINE) implementation

Author
Krishna Vedala

Structure of an ADALINE network. Source: Wikipedia

ADALINE is one of the first and simplest single layer artificial neural network. The algorithm essentially implements a linear function

\[ f\left(x_0,x_1,x_2,\ldots\right) = \sum_j x_jw_j+\theta \]

where \(x_j\) are the input features of a sample, \(w_j\) are the coefficients of the linear function and \(\theta\) is a constant. If we know the \(w_j\), then for any given set of features, \(y\) can be computed. Computing the \(w_j\) is a supervised learning algorithm wherein a set of features and their corresponding outputs are given and weights are computed using stochastic gradient descent method.

Function Documentation

◆ main()

int main ( int  argc,
char **  argv 
)

Main function

357  {
358  std::srand(std::time(nullptr)); // initialize random number generator
359 
360  double eta = 0.1; // default value of eta
361  if (argc == 2) { // read eta value from commandline argument if present
362  eta = strtof(argv[1], nullptr);
363  }
364 
365  test1(eta);
366 
367  std::cout << "Press ENTER to continue..." << std::endl;
368  std::cin.get();
369 
370  test2(eta);
371 
372  std::cout << "Press ENTER to continue..." << std::endl;
373  std::cin.get();
374 
375  test3(eta);
376 
377  return 0;
378 }
Here is the call graph for this function:

◆ test1()

void test1 ( double  eta = 0.01)

test function to predict points in a 2D coordinate system above the line \(x=y\) as +1 and others as -1. Note that each point is defined by 2 values or 2 features.

Parameters
[in]etalearning rate (optional, default=0.01)
224  {
225  adaline ada(2, eta); // 2 features
226 
227  const int N = 10; // number of sample points
228 
230  std::vector<double>({0, 1}), std::vector<double>({1, -2}),
231  std::vector<double>({2, 3}), std::vector<double>({3, -1}),
232  std::vector<double>({4, 1}), std::vector<double>({6, -5}),
233  std::vector<double>({-7, -3}), std::vector<double>({-8, 5}),
234  std::vector<double>({-9, 2}), std::vector<double>({-10, -15})};
235  std::array<int, N> y = {1, -1, 1, -1, -1,
236  -1, 1, 1, 1, -1}; // corresponding y-values
237 
238  std::cout << "------- Test 1 -------" << std::endl;
239  std::cout << "Model before fit: " << ada << std::endl;
240 
241  ada.fit<N>(X, y);
242  std::cout << "Model after fit: " << ada << std::endl;
243 
244  int predict = ada.predict({5, -3});
245  std::cout << "Predict for x=(5,-3): " << predict;
246  assert(predict == -1);
247  std::cout << " ...passed" << std::endl;
248 
249  predict = ada.predict({5, 8});
250  std::cout << "Predict for x=(5,8): " << predict;
251  assert(predict == 1);
252  std::cout << " ...passed" << std::endl;
253 }
Here is the call graph for this function:

◆ test2()

void test2 ( double  eta = 0.01)

test function to predict points in a 2D coordinate system above the line \(x+3y=-1\) as +1 and others as -1. Note that each point is defined by 2 values or 2 features. The function will create random sample points for training and test purposes.

Parameters
[in]etalearning rate (optional, default=0.01)
262  {
263  adaline ada(2, eta); // 2 features
264 
265  const int N = 50; // number of sample points
266 
268  std::array<int, N> Y{}; // corresponding y-values
269 
270  // generate sample points in the interval
271  // [-range2/100 , (range2-1)/100]
272  int range = 500; // sample points full-range
273  int range2 = range >> 1; // sample points half-range
274  for (int i = 0; i < N; i++) {
275  double x0 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
276  double x1 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
277  X[i] = std::vector<double>({x0, x1});
278  Y[i] = (x0 + 3. * x1) > -1 ? 1 : -1;
279  }
280 
281  std::cout << "------- Test 2 -------" << std::endl;
282  std::cout << "Model before fit: " << ada << std::endl;
283 
284  ada.fit(X, Y);
285  std::cout << "Model after fit: " << ada << std::endl;
286 
287  int N_test_cases = 5;
288  for (int i = 0; i < N_test_cases; i++) {
289  double x0 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
290  double x1 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
291 
292  int predict = ada.predict({x0, x1});
293 
294  std::cout << "Predict for x=(" << x0 << "," << x1 << "): " << predict;
295 
296  int expected_val = (x0 + 3. * x1) > -1 ? 1 : -1;
297  assert(predict == expected_val);
298  std::cout << " ...passed" << std::endl;
299  }
300 }
Here is the call graph for this function:

◆ test3()

void test3 ( double  eta = 0.01)

test function to predict points in a 3D coordinate system lying within the sphere of radius 1 and centre at origin as +1 and others as -1. Note that each point is defined by 3 values but we use 6 features. The function will create random sample points for training and test purposes. The sphere centred at origin and radius 1 is defined as: \(x^2+y^2+z^2=r^2=1\) and if the \(r^2<1\), point lies within the sphere else, outside.

Parameters
[in]etalearning rate (optional, default=0.01)
313  {
314  adaline ada(6, eta); // 2 features
315 
316  const int N = 100; // number of sample points
317 
319  std::array<int, N> Y{}; // corresponding y-values
320 
321  // generate sample points in the interval
322  // [-range2/100 , (range2-1)/100]
323  int range = 200; // sample points full-range
324  int range2 = range >> 1; // sample points half-range
325  for (int i = 0; i < N; i++) {
326  double x0 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
327  double x1 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
328  double x2 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
329  X[i] = std::vector<double>({x0, x1, x2, x0 * x0, x1 * x1, x2 * x2});
330  Y[i] = ((x0 * x0) + (x1 * x1) + (x2 * x2)) <= 1.f ? 1 : -1;
331  }
332 
333  std::cout << "------- Test 3 -------" << std::endl;
334  std::cout << "Model before fit: " << ada << std::endl;
335 
336  ada.fit(X, Y);
337  std::cout << "Model after fit: " << ada << std::endl;
338 
339  int N_test_cases = 5;
340  for (int i = 0; i < N_test_cases; i++) {
341  double x0 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
342  double x1 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
343  double x2 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
344 
345  int predict = ada.predict({x0, x1, x2, x0 * x0, x1 * x1, x2 * x2});
346 
347  std::cout << "Predict for x=(" << x0 << "," << x1 << "," << x2
348  << "): " << predict;
349 
350  int expected_val = ((x0 * x0) + (x1 * x1) + (x2 * x2)) <= 1.f ? 1 : -1;
351  assert(predict == expected_val);
352  std::cout << " ...passed" << std::endl;
353  }
354 }
Here is the call graph for this function:
std::srand
T srand(T... args)
std::vector< double >
test1
void test1(double eta=0.01)
Definition: adaline_learning.cpp:224
std::cout
std::array
STL class.
std::rand
T rand(T... args)
std::endl
T endl(T... args)
std::strtof
T strtof(T... args)
std::time
T time(T... args)
std::cin
test2
void test2(double eta=0.01)
Definition: adaline_learning.cpp:262
test3
void test3(double eta=0.01)
Definition: adaline_learning.cpp:313
machine_learning::adaline
Definition: adaline_learning.cpp:46