开源的人工神经网络计算库 FANN 学习笔记 1

这年头机器学习非常的火 神经网络算是机器学习算法中的比较重要的一种。这段时间我也花了些功夫 学了点皮毛 顺便做点学习笔记。

介绍人工神经网络的基本理论的教科书很多。我正在看的是蒋宗礼教授写的《人工神经网络导论》 之所以选这本书 主要是这本比较薄 太厚的书实在是啃不动。这本书写的也比较浅显 用来入门正合适。

看书的同时也在网上找了找人工神经网络的库代码。感觉 FANN 这个库还不错 就顺道学了学这个库的使用方法。

FANN 是个开源的 C 语言实现的人工神经网络库 由于是标准 C 语言写成的 所以对操作系统等的要求很少 在各个平台下都可以运行。而且这个库支持定点运算 在没有浮点处理器的 CPU 上运行会比别的不支持定点运算的库快很多。

FANN 虽然是纯 C 语言写成的 但是按照面向对象的思想构架的 接口设计的很好。有较为详细的文档 用起来很方便。而且已经支持在20多种编程语言环境下使用 比如 C#、 JAVA、Delphi、PYTHON、PHP、PERL、RUBY、Javascript、Matlab、R 等。

下面是一个非常简单的例子。我们用个神经网络来模拟两个布尔变量的与运算。我们的训练数据放到一个文件中 文件名为”and.data”。

内容如下

4 2 1
0 0
0
0 1
0
1 0
0
1 1
1

其中 4 2 1 表示我们的训练数据集有 4 条训练数据。每个数据有 2 个输入和 1 个输出。之后跟的就是输入与输出数据。

我们知道与运算可以用单层感知器来实现。也就是说可以用 2 层的神经网络来实现 1层输入层、1层输出层 。

#include doublefann.h const unsigned int NUM_INPUT 2;const unsigned int NUM_OUTPUT 1;const unsigned int NUM_LAYERS 2;const unsigned int NUM_NEURONS_HIDDEN 1;const float DESIRED_ERROR (const float) 0.0001;const unsigned int MAX_EPOCHS 1000;const unsigned int EPOCHS_BETWEEN_REPORTS 10;int main(int argc, char *argv[]){ struct fann *ann; struct fann_train_data *data; printf( Creating network.\\n ); ann fann_create_standard(NUM_LAYERS, NUM_INPUT, NUM_NEURONS_HIDDEN, NUM_OUTPUT); data fann_read_train_from_file( q:\\\\and.data ); printf( Training network.\\n ); fann_train_on_data(ann, data, MAX_EPOCHS, EPOCHS_BETWEEN_REPORTS, DESIRED_ERROR); printf( Testing network. %f\\n , fann_test_data(ann, data));// fann_save(ann, q:\\\\and_float.net fann_type input[2]; fann_type *calc_out; input[0] 0; input[1] 0; calc_out fann_run(ann, input); printf( and test (%f,%f) - %f\\n , input[0], input[1], calc_out[0]); input[0] 0; input[1] 1; calc_out fann_run(ann, input); printf( and test (%f,%f) - %f\\n , input[0], input[1], calc_out[0]); input[0] 1; input[1] 0; calc_out fann_run(ann, input); printf( and test (%f,%f) - %f\\n , input[0], input[1], calc_out[0]); input[0] 1; input[1] 1; calc_out fann_run(ann, input); printf( and test (%f,%f) - %f\\n , input[0], input[1], calc_out[0]);}

程序很简单 有一行被我注释掉了

fann_save(ann, q:\\\\and_float.net );

这个是用来将训练好的神经网络保存起来。以便后面使用。实际上 对于大多数的神经网络应用来说 网络的训练过程与网络的使用都是分开的。因为通常训练神经网络是很费时间的 但是一旦训练好了 就可以一直使用。没有必要每次用之前再训练一次。

一旦将fann 的状态保存起来了。后面在使用就很方便。比如下面这样。

 struct fann *ann; ann fann_create_from_file( q:\\\\and_float.net );

运行后输出的结果如下

Creating network.
Training network.
Max epochs 1000. Desired error: 0.0001000000.
Epochs 1. Current error: 0.2458046824. Bit fail 4.
Epochs 10. Current error: 0.1185930669. Bit fail 2.
Epochs 20. Current error: 0.0314348340. Bit fail 0.
Epochs 30. Current error: 0.0145781031. Bit fail 0.
Epochs 40. Current error: 0.0055742161. Bit fail 0.
Epochs 50. Current error: 0.0024458752. Bit fail 0.
Epochs 60. Current error: 0.0015401742. Bit fail 0.
Epochs 70. Current error: 0.0008485225. Bit fail 0.
Epochs 80. Current error: 0.0004349701. Bit fail 0.
Epochs 90. Current error: 0.0002433052. Bit fail 0.
Epochs 100. Current error: 0.0001541690. Bit fail 0.
Epochs 103. Current error: 0.0000989792. Bit fail 0.
Testing network. 0.000095
and test (0.000000,0.000000) - 0.000000
and test (0.000000,1.000000) - 0.012569
and test (1.000000,0.000000) - 0.007818
and test (1.000000,1.000000) - 0.987361

可以看到拟合的结果非常好。我们还知道单级感知器无法表达异或运算。下面就可以做个实验 将与运算改为异或运算。训练数据如下

4 2 10 000 111 011 10

程序基本不需要变。运行结果是这样的。

Creating network.
Training network.
Max epochs 1000. Desired error: 0.0001000000.
Epochs 1. Current error: 0.2500112057. Bit fail 4.
Epochs 10. Current error: 0.2502280176. Bit fail 4.
Epochs 20. Current error: 0.2500012517. Bit fail 4.
Epochs 30. Current error: 0.2500000000. Bit fail 4.
Epochs 40. Current error: 0.2500000000. Bit fail 4.
Epochs 50. Current error: 0.2500000000. Bit fail 4.
Epochs 60. Current error: 0.2500000000. Bit fail 4.
Epochs 70. Current error: 0.2500000000. Bit fail 4.
Epochs 80. Current error: 0.2500000000. Bit fail 4.
Epochs 90. Current error: 0.2500000000. Bit fail 4.
Epochs 100. Current error: 0.2500000298. Bit fail 4.
Epochs 110. Current error: 0.2500000000. Bit fail 4.
Epochs 120. Current error: 0.2500000000. Bit fail 4.
Epochs 130. Current error: 0.2500000000. Bit fail 4.
Epochs 140. Current error: 0.2499999851. Bit fail 4.
Epochs 150. Current error: 0.2500000000. Bit fail 4.
Epochs 160. Current error: 0.2500000000. Bit fail 4.
Epochs 170. Current error: 0.2500000000. Bit fail 4.
Epochs 180. Current error: 0.2500000000. Bit fail 4.
Epochs 190. Current error: 0.2500000000. Bit fail 4.
Epochs 200. Current error: 0.2500000000. Bit fail 4.
Epochs 210. Current error: 0.2500000000. Bit fail 4.
Epochs 220. Current error: 0.2500000000. Bit fail 4.
Epochs 230. Current error: 0.2500000000. Bit fail 4.
Epochs 240. Current error: 0.2500000000. Bit fail 4.
Epochs 250. Current error: 0.2500000000. Bit fail 4.
Epochs 260. Current error: 0.2500000000. Bit fail 4.
Epochs 270. Current error: 0.2500000000. Bit fail 4.
Epochs 280. Current error: 0.2500000000. Bit fail 4.
Epochs 290. Current error: 0.2500000000. Bit fail 4.
Epochs 300. Current error: 0.2500000000. Bit fail 4.
Epochs 310. Current error: 0.2500000000. Bit fail 4.
Epochs 320. Current error: 0.2500000000. Bit fail 4.
Epochs 330. Current error: 0.2500000000. Bit fail 4.
Epochs 340. Current error: 0.2500000000. Bit fail 4.
Epochs 350. Current error: 0.2500000000. Bit fail 4.
Epochs 360. Current error: 0.2500000000. Bit fail 4.
Epochs 370. Current error: 0.2500000000. Bit fail 4.
Epochs 380. Current error: 0.2500000000. Bit fail 4.
Epochs 390. Current error: 0.2500000000. Bit fail 4.
Epochs 400. Current error: 0.2500000000. Bit fail 4.
Epochs 410. Current error: 0.2500000298. Bit fail 4.
Epochs 420. Current error: 0.2500000000. Bit fail 4.
Epochs 430. Current error: 0.2500000000. Bit fail 4.
Epochs 440. Current error: 0.2500000000. Bit fail 4.
Epochs 450. Current error: 0.2499999851. Bit fail 4.
Epochs 460. Current error: 0.2500000000. Bit fail 4.
Epochs 470. Current error: 0.2500000000. Bit fail 4.
Epochs 480. Current error: 0.2500000000. Bit fail 4.
Epochs 490. Current error: 0.2500000000. Bit fail 4.
Epochs 500. Current error: 0.2500000000. Bit fail 4.
Epochs 510. Current error: 0.2500000000. Bit fail 4.
Epochs 520. Current error: 0.2500000000. Bit fail 4.
Epochs 530. Current error: 0.2500000000. Bit fail 4.
Epochs 540. Current error: 0.2500000000. Bit fail 4.
Epochs 550. Current error: 0.2500000000. Bit fail 4.
Epochs 560. Current error: 0.2500000000. Bit fail 4.
Epochs 570. Current error: 0.2500000000. Bit fail 4.
Epochs 580. Current error: 0.2500000000. Bit fail 4.
Epochs 590. Current error: 0.2500000000. Bit fail 4.
Epochs 600. Current error: 0.2500000000. Bit fail 4.
Epochs 610. Current error: 0.2500000000. Bit fail 4.
Epochs 620. Current error: 0.2500000000. Bit fail 4.
Epochs 630. Current error: 0.2500000000. Bit fail 4.
Epochs 640. Current error: 0.2500000000. Bit fail 4.
Epochs 650. Current error: 0.2500000000. Bit fail 4.
Epochs 660. Current error: 0.2500000000. Bit fail 4.
Epochs 670. Current error: 0.2500000000. Bit fail 4.
Epochs 680. Current error: 0.2500000000. Bit fail 4.
Epochs 690. Current error: 0.2500000000. Bit fail 4.
Epochs 700. Current error: 0.2500000000. Bit fail 4.
Epochs 710. Current error: 0.2500000000. Bit fail 4.
Epochs 720. Current error: 0.2500000298. Bit fail 4.
Epochs 730. Current error: 0.2500000000. Bit fail 4.
Epochs 740. Current error: 0.2500000000. Bit fail 4.
Epochs 750. Current error: 0.2500000000. Bit fail 4.
Epochs 760. Current error: 0.2499999851. Bit fail 4.
Epochs 770. Current error: 0.2500000000. Bit fail 4.
Epochs 780. Current error: 0.2500000000. Bit fail 4.
Epochs 790. Current error: 0.2500000000. Bit fail 4.
Epochs 800. Current error: 0.2500000000. Bit fail 4.
Epochs 810. Current error: 0.2500000000. Bit fail 4.
Epochs 820. Current error: 0.2500000000. Bit fail 4.
Epochs 830. Current error: 0.2500000000. Bit fail 4.
Epochs 840. Current error: 0.2500000000. Bit fail 4.
Epochs 850. Current error: 0.2500000000. Bit fail 4.
Epochs 860. Current error: 0.2500000000. Bit fail 4.
Epochs 870. Current error: 0.2500000000. Bit fail 4.
Epochs 880. Current error: 0.2500000000. Bit fail 4.
Epochs 890. Current error: 0.2500000000. Bit fail 4.
Epochs 900. Current error: 0.2500000000. Bit fail 4.
Epochs 910. Current error: 0.2500000000. Bit fail 4.
Epochs 920. Current error: 0.2500000000. Bit fail 4.
Epochs 930. Current error: 0.2500000000. Bit fail 4.
Epochs 940. Current error: 0.2500000000. Bit fail 4.
Epochs 950. Current error: 0.2500000000. Bit fail 4.
Epochs 960. Current error: 0.2500000000. Bit fail 4.
Epochs 970. Current error: 0.2500000000. Bit fail 4.
Epochs 980. Current error: 0.2500000000. Bit fail 4.
Epochs 990. Current error: 0.2500000000. Bit fail 4.
Epochs 1000. Current error: 0.2500000000. Bit fail 4.
Testing network. 0.250000
xor test (0.000000,0.000000) - 0.500042
xor test (0.000000,1.000000) - 0.500058
xor test (1.000000,0.000000) - 0.500064
xor test (1.000000,1.000000) - 0.500080

可以看到训练无法收敛。如果将神经网络改为 3 层 就具备表达异或的能力了。

程序只需改两行

const unsigned int NUM_LAYERS 3;const unsigned int NUM_NEURONS_HIDDEN 2;

这两行的意思是将网络层数改为 3 中间那层包含 2 个神经元。之后运行的结果如下

Creating network.
Training network.
Max epochs 1000. Desired error: 0.0001000000.
Epochs 1. Current error: 0.2500394285. Bit fail 4.
Epochs 10. Current error: 0.2503248155. Bit fail 4.
Epochs 20. Current error: 0.2500005960. Bit fail 4.
Epochs 30. Current error: 0.2500000596. Bit fail 4.
Epochs 40. Current error: 0.2500004768. Bit fail 4.
Epochs 50. Current error: 0.2487613261. Bit fail 4.
Epochs 60. Current error: 0.2448616028. Bit fail 4.
Epochs 70. Current error: 0.2345527709. Bit fail 4.
Epochs 80. Current error: 0.1928165257. Bit fail 2.
Epochs 90. Current error: 0.0843421519. Bit fail 1.
Epochs 100. Current error: 0.0168493092. Bit fail 0.
Epochs 110. Current error: 0.0046176752. Bit fail 0.
Epochs 120. Current error: 0.0023012348. Bit fail 0.
Epochs 130. Current error: 0.0014233238. Bit fail 0.
Epochs 140. Current error: 0.0008981032. Bit fail 0.
Epochs 150. Current error: 0.0005876040. Bit fail 0.
Epochs 160. Current error: 0.0003150564. Bit fail 0.
Epochs 170. Current error: 0.0001641331. Bit fail 0.
Epochs 175. Current error: 0.0000839768. Bit fail 0.
Testing network. 0.000091
xor test (0.000000,0.000000) - 0.011072
xor test (0.000000,1.000000) - 0.993215
xor test (1.000000,0.000000) - 0.992735
xor test (1.000000,1.000000) - 0.011975

结果非常的好。