zl程序教程

您现在的位置是:首页 >  后端

当前栏目

ubuntu20 caffe opencv4 conda python3.7.10

python3.7conda 10 Caffe
2023-09-14 09:06:46 时间

如果你只想要pycaffe,那可以conda+python3.7.10,然后conda install caffe-gpu

环境

ubuntu20
cuda11.7 cudnn8(不过caffe似乎不能加速cudnn8
conda python3.7.10
假设你已经装了opencv4,cuda11.7,cudnn8
opencv4安装教程
cuda和cudnn安装教程
建议ubuntu和anaconda,pip都换个源(ubuntu换源anaconda换源,建议ustc/bfsu)

sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler -y
sudo apt-get install --no-install-recommends libboost-all-dev -y
sudo apt-get install libopenblas-dev liblapack-dev libatlas-base-dev -y
sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev -y
sudo apt-get install git cmake build-essential -y

python

conda create -n caffe_env python==3.7.10 pip setuptools
conda activate caffe_env
pip install numpy scipy scikit-image matplotlib ipython h5py leveldb networkx nose pandas python-dateutil protobuf python-gflags pyyaml pillow six google-api-python-client

conda install pydot graphviz boost

这里用conda装boost是因为apt似乎只会装和系统python一样的版本的boost

安装

https://github.com/BVLC/caffe

tar -zxvf caffe-master.zip
cd caffe-master
cp Makefile.config.example Makefile.config

修改Makefile.config

  修改代码:
#USE_OPENCV := 0
#OPENCV_VERSION := 3
#WITH_PYTHON_LAYER := 1
  为:
USE_OPENCV := 1
OPENCV_VERSION := 4
WITH_PYTHON_LAYER := 1
 
  修改代码:
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
  为:
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial /usr/local/include/opencv4
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial
  # 最后是自己的opencv的include和bin路径,解决fatal error: opencv2/core/core.hpp问题
 
 修改代码:
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
		# $(ANACONDA_HOME)/include/python2.7 \
		# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include
  为:
ANACONDA_HOME := $(HOME)/miniconda3/envs/caffe_env
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
                $(ANACONDA_HOME)/include/python3.7m \
                $(ANACONDA_HOME)/lib/python3.7/site-packages/numpy/core/include
  # 根据自己的路径修改
 

 修改代码:
# PYTHON_LIBRARIES := boost_python3 python3.5m

 修改代码:
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
		# $(ANACONDA_HOME)/include/python2.7 \
		# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include
  为:
PYTHON_LIBRARIES := boost_python37 python3.7m
  # 根据自己的路径修改


  修改代码:
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
		-gencode arch=compute_20,code=sm_21 \
		-gencode arch=compute_30,code=sm_30 \
		-gencode arch=compute_35,code=sm_35 \
		-gencode arch=compute_50,code=sm_50 \
		-gencode arch=compute_52,code=sm_52 \
		-gencode arch=compute_60,code=sm_60 \
		-gencode arch=compute_61,code=sm_61 \
		-gencode arch=compute_61,code=compute_61
  为:
CUDA_ARCH := -gencode arch=compute_50,code=sm_50 \
                -gencode arch=compute_52,code=sm_52 \
                -gencode arch=compute_60,code=sm_60 \
                -gencode arch=compute_61,code=sm_61 \
                -gencode arch=compute_61,code=compute_61
  # 根据自己的架构和版本修改,可参考https://arnon.dk/matching-sm-architectures-arch-and-gencode-for-various-nvidia-cards/

修改Makefile

修改Makefile

  修改代码:
NVCCFLAGS +=-ccbin=$(CXX) -Xcompiler-fPIC $(COMMON_FLAGS)
  为:
NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)
 
  修改代码:
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5
  为:
LIBRARIES += glog gflags protobuf boost_system boost_filesystem hdf5_hl hdf5 m opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs
  # 解决undefined reference to cv::imread(cv::String const&, int)...,可参考https://github.com/BVLC/caffe/issues/2348
 
  修改代码:
PYTHON_LIBRARIES ?= boost_python python2.7
  为:
PYTHON_LIBRARIES ?= boost_python37 python3.7
  # 根据自己的版本修改

修改src/caffe/util/io.cpp

注意这里要修改的可能不止一处

 将:
CV_LOAD_IMAGE_COLOR
 修改为:
cv::IMREAD_COLOR


 将:
CV_LOAD_IMAGE_GRAYSCALE
 修改为:
cv::IMREAD_GRAYSCALE

修改src/caffe/layers/window_data_layer.cpp

注意这里要修改的可能不止一处

 将:
CV_LOAD_IMAGE_COLOR
 修改为:
cv::IMREAD_COLOR


 将:
CV_LOAD_IMAGE_GRAYSCALE
 修改为:
cv::IMREAD_GRAYSCALE

修改src/caffe/data_transformer.cpp

  修改代码:
#ifdef USE_OPENCV
#include <opencv2/core/core.hpp>
#endif  // USE_OPENCV
  为:
#ifdef USE_OPENCV
#include <opencv2/core/core.hpp>
#include <opencv2/core/mat.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/highgui/highgui_c.h>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/imgproc/imgproc_c.h>
#include <opencv2/opencv.hpp>
#endif  // USE_OPENCV

include/caffe/common.hpp

  新增代码:
// Supporting OpenCV4
#if (CV_MAJOR_VERSION == 4)
#define CV_LOAD_IMAGE_COLOR cv::IMREAD_COLOR
#define CV_LOAD_IMAGE_GRAYSCALE cv::IMREAD_GRAYSCALE
#endif
 
  以下为原代码(用于定位):
// See PR #1236
namespace cv { class Mat; }

编译

先切一下环境(也许不需要)

conda activate caffe_env
make all -j8

测试

先切一下环境(也许不需要)

conda activate caffe_env

这里是因为boost_python是conda装的,默认不在系统里,所以先改一下LD_LIBRARY_PATH

export LD_LIBRARY_PATH="/home/icml/miniconda3/envs/caffe_env/lib:$LD_LIBRARY_PATH"
make runtest -j8

在这里插入图片描述

编译pycaffe

先切一下环境(也许不需要)

conda activate caffe_env

需要的python包在最前面创建python的时候就已经安装了

make pycaffe -j8

测试

先切一下环境

conda activate caffe_env

需要的python包在最前面创建python的时候就已经安装了

这里是因为boost_python是conda装的,默认不在系统里,所以先改一下LD_LIBRARY_PATH

export LD_LIBRARY_PATH="/home/icml/miniconda3/envs/caffe_env/lib:$LD_LIBRARY_PATH"

然后

export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION='python' 

这一句是为了防止出现下面这个错误
在这里插入图片描述
开始测试

make pytest -j8

在这里插入图片描述

修改环境变量

在~/.bashrc的后面加入

export LD_LIBRARY_PATH="/home/icml/miniconda3/envs/caffe_env/lib:$LD_LIBRARY_PA>
export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION='python' 
export PYTHONPATH="${PYTHONPATH}:/mnt/data/caffe-master/python"

然后

source ~/.bashrc

如果你有zsh,就再做一遍类似的操作

测试

C++

mnist

首先cd进入caffe的目录

sh data/mnist/get_mnist.sh
sh examples/mnist/create_mnist.sh
nano examples/mnist/lenet_solver.prototxt

修改最后的max_iter和solver_mode(虽然我没改

开始训练

time sh examples/mnist/train_lenet.sh

花了1min,还行把
在这里插入图片描述

https://blog.csdn.net/ynzzxc/article/details/116780168
https://blog.csdn.net/yhaolpz/article/details/71375762
https://qengineering.eu/install-caffe-on-ubuntu-20.04-with-opencv-4.4.html
https://stackoverflow.com/questions/50839667/protofile-proto-a-file-with-this-name-is-already-in-the-pool