airflow安装教程(local模式)
2023-06-13 09:12:44 时间
环境介绍
OS:centos 7.6
python:3.6.8
pip:21.2.4
具体安装
pip配置阿里云清华源
mkdir -p $HOME/.config/pip/
vim $HOME/.config/pip/pip.conf
新增如下内容:
[global]
index-url = https://pypi.tuna.tsinghua.edu.cn/simple
执行安装:
export AIRFLOW_HOME=/data/airflow
AIRFLOW_VERSION=2.2.2
export PYTHON_VERSION=3.6
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
pip install virtualenv
启动airflow
airflow standalone
报错
[root@hadoop1 ~]# airflow standalone
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 5, in <module>
from airflow.__main__ import main
File "/usr/local/lib/python3.6/site-packages/airflow/__init__.py", line 34, in <module>
from airflow import settings
File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 35, in <module>
from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf # NOQA F401
File "/usr/local/lib/python3.6/site-packages/airflow/configuration.py", line 1129, in <module>
conf.validate()
File "/usr/local/lib/python3.6/site-packages/airflow/configuration.py", line 224, in validate
self._validate_config_dependencies()
File "/usr/local/lib/python3.6/site-packages/airflow/configuration.py", line 278, in _validate_config_dependencies
f"error: sqlite C library version too old (< {min_sqlite_version}). "
airflow.exceptions.AirflowConfigException: error: sqlite C library version too old (< 3.15.0). See https://airflow.apache.org/docs/apache-airflow/2.2.2/howto/set-up-database.html#setting-up-a-sqlite-database
解决方法:
参考:Set up a Database Backend — Airflow Documentation (apache.org),重新配置sqlite
wget https://www.sqlite.org/src/tarball/sqlite.tar.gz
tar xzf sqlite.tar.gz
cd sqlite/
export CFLAGS="-DSQLITE_ENABLE_FTS3 \
-DSQLITE_ENABLE_FTS3_PARENTHESIS \
-DSQLITE_ENABLE_FTS4 \
-DSQLITE_ENABLE_FTS5 \
-DSQLITE_ENABLE_JSON1 \
-DSQLITE_ENABLE_LOAD_EXTENSION \
-DSQLITE_ENABLE_RTREE \
-DSQLITE_ENABLE_STAT4 \
-DSQLITE_ENABLE_UPDATE_DELETE_LIMIT \
-DSQLITE_SOUNDEX \
-DSQLITE_TEMP_STORE=3 \
-DSQLITE_USE_URI \
-O2 \
-fPIC"
export PREFIX="/usr/local"
LIBS="-lm" ./configure --disable-tcl --enable-shared --enable-tempstore=always --prefix="$PREFIX"
make
make install
将
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
新增到/etc/profile中,并执行source /etc/profile生效
再次启动airflow
airflow standalone
可以看到:
webserver | ____________ _____________
webserver | ____ |__( )_________ __/__ /________ __
webserver | ____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
webserver | ___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
webserver | _/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
webserver | [2021-12-03 03:36:04,583] {dagbag.py:500} INFO - Filling up the DagBag from /dev/null
webserver | [2021-12-03 03:36:04,851] {manager.py:512} WARNING - Refused to delete permission view, assoc with role exists DAG Runs.can_create Admin
webserver | [2021-12-03 03:36:07 -0500] [5428] [INFO] Starting gunicorn 20.1.0
webserver | [2021-12-03 03:36:07 -0500] [5428] [INFO] Listening at: http://0.0.0.0:8080 (5428)
webserver | [2021-12-03 03:36:07 -0500] [5428] [INFO] Using worker: sync
webserver | [2021-12-03 03:36:07 -0500] [5433] [INFO] Booting worker with pid: 5433
webserver | [2021-12-03 03:36:07 -0500] [5434] [INFO] Booting worker with pid: 5434
webserver | [2021-12-03 03:36:07 -0500] [5435] [INFO] Booting worker with pid: 5435
webserver | [2021-12-03 03:36:07 -0500] [5436] [INFO] Booting worker with pid: 5436
进入$AIRFLOW_HOME查看admin密码
cat $AIRFLOW_HOME/standalone_admin_password.txt
但是请注意,SQLite只适合测试场景,不能使用其作为生产环境部署使用,生产环境部署建议使用Postgres或者MySQL。
0 0 投票数
文章评分
本文为从大数据到人工智能博主「xiaozhch5」的原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接及本声明。
原文链接:https://lrting.top/backend/8410/
相关文章
- 利用pycharm安装requests库「建议收藏」
- Java:Eclipse下载安装教程,以及Eclipse 安装汉化包的方法[通俗易懂]
- Android Studio 安装教程
- Node.js最新最详细安装教程(2020)
- java编译器——安装IDEA 步骤
- PR 2023 v23.0 下载Adobe Premiere Pro 2023安装教程
- Thinkphp6.0学习(1)安装tp6与测试
- AutoGluon GPU 版本 安装配置教程
- ch-Character Animator 2022 下载安装教程讲解+各版本安装获取
- 【Windows+mac】Acrobat Pro 2021 PDF编辑器,全版本安装包下载以及安装教程
- Audition 2019 For Mac软件安装教程 AU软件全版本下载
- Photoshop CC2017软件安装教程(支持32位/64位)--PS软件全版本
- Adobe Acrobat 9 Pro软件安装教程(强大的PDF编辑软件)PDF编辑器全版本下载
- AutoCAD 2023 | MAC+WIN 中文版安装教程
- wordpress建站怎么安装?WP建站教程
- office 2007软件安装教程办公软件安装-office全版本软件下载地址
- office办公软件介绍---office2021版本下载win系统安装教程
- Spss中文版27电脑版安装,统计分析Spss软件下载安装激活教程
- PyCharm下载:Python编程利器PyCharm 2022版安装激活汉化教程
- Adobe Premiere pro 2023软件安装教程-PR软件全版本下载地址
- Adobe Premiere 2020 软件安装教程-PR软件全版本下载地址
- ftpd安装Linux下的VSFTPD服务器(linux安装vs)
- 更好MySQL:安装哪个版本最佳?(mysql安装哪个版本)
- 安装Linux系统:改变电脑的一次新挑战(电脑如何装linux系统)
- 搭建强大全面的数据库——阿里云安装MySQL(阿里云安装mysql)
- U盘启动Linux:无需安装!(u盘引导linux)
- 教程 | 如何在Linux系统上安装微信?(linux安装微信)
- Linux安装上火狐浏览器:操作简单,享受安全体验(linux装火狐)
- CentOS 7安装MySQL教程(cos7安装mysql)
- CDH安装ORACLE教程步步深入掌握(CDH安装ORACLE)
- 实用教程Oracle 9i 安装必备指南(oracle92安装教程)
- Oracle 13安装配置指南(oracle13配置要求)