在Matlab中使用Caffe出现HDF5 library version mismatched error的解决办法

前言

最近用Caffe训练了一个CNN,输入训练数据是HDF5格式的,在使用Matlab读取网络并且做测试时,出现了奇怪的错误,“HDF5 library version mismatched error”。折腾了半天终于解决,留存一下。

具体错误信息如下:

1
Warning! ***HDF5 library version mismatched error***
The HDF5 header files used to compile this application do not match
the version used by the HDF5 library to which this application is linked.
Data corruption or segmentation faults may occur if the application continues.
This can happen when an application was compiled by one version of HDF5 but
linked with a different version of static or shared HDF5 library.
You should recompile the application or check your shared library related
settings such as 'LD_LIBRARY_PATH'.
You can, at your own risk, disable this warning by setting the environment
variable 'HDF5_DISABLE_VERSION_CHECK' to a value of '1'.
Setting it to 2 or higher will suppress the warning messages totally.
Headers are 1.8.11, library is 1.8.12

定位HDF5

从出错信息可以判断,有两个不同版本的HDF5被安装在电脑上,分别为1.8.11和1.8.12,在编译库文件时所用的版本跟现在所引用的头文件版本不一致。首先找出HDF5的安装路径:

1
locate hdf5

从输出的结果看到,主要有两个地方存在HDF5,一个是Matlab的安装目录,一个是/usr/lib/x86_64-linux-gnu。其中,后面的目录下能找到名为libhdf5.settings的文件,包含了该目录下HDF5的基本信息,可以看到这是1.8.11版本的。

1
General Information:
-------------------
		   HDF5 Version: 1.8.11
		  Configured on: Mon Dec 23 00:29:04 UTC 2013
		  Configured by: buildd@lamiak
		 Configure mode: production
		    Host system: x86_64-pc-linux-gnu
	      Uname information: Linux lamiak 3.2.0-37-generic #58-Ubuntu SMP Thu Jan 24 15:28:10 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
		       Byte sex: little-endian
		      Libraries: static, shared
	     Installation point: /usr

没有找到Matlab下的HDF5的版本信息,不过既然系统原来安装的是1.8.11,为了解决兼容性问题,需要手动重新安装HDF5 1.8.12版本。

安装HDF5 v1.8.12

HDF网站下载1.8.12版本的源代码。
解压,按照release_docs/INSTALL上的说明进行编译安装即可。

1
2
3
4
5
6
$ cd hdf5-X.Y.Z
$ ./configure --prefix=/usr/local/hdf5 <more configure_flags>
$ make
$ make check # run test suite.
$ make install
$ make check-install # verify installation.

安装成功后需要添加环境变量

1
vim ~/.bashrc

在文件末加入下面这一行:

1
export LD_LIBRARY_PATH=/usr/local/hdf5/lib:$LD_LIBRARY_PATH

然后执行下列语句使得修改即时生效:

1
source ~/.bashrc

重新编译Caffe

首先修改Makefile.config,更新HDF5安装目录。搜索hdf5,替换成新目录即可:

1
2
3
# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/local/hdf5/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/local/hdf5/lib

重新编译Caffe:

1
2
3
4
5
6
7
make clean
make all -j8
make test -j8
make runtest -j8

make pycaffe
make matcaffe

启动Matlab

此时用命令行启动Matlab

1
/usr/local/Matlab/R2015b/bin/Matlab

如果已经把Matlab的bin目录添加到环境变量中,直接在命令行输入matlab即可。

此时重新运行代码,载入caffe网络,之前的问题就不再出现了。