sqoop的安装
安装 Sqoop
2023-09-11 14:22:24 时间
Sqoop是一个用来完成Hadoop和关系型数据库中的数据相互转移的工具,
他可以将关系型数据库(MySql,Oracle,Postgres等)中的数据导入Hadoop的HDFS中,
也可以将HDFS的数据导入关系型数据库中。
前期准备:
1.Sqoop需要安装在成功部署的Hadoop平台上,并且要求Hadoop已经正常启动。hadoop集群的搭建
2.安装了mysql数据库,并且已经启动.CentOS安装mysql源码包
3.下载mysql-connector-java-5.1.12.tar.gz。
安装过程;
1.下载sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz压缩包。
2.上传至/usr/local/src
3.解压缩并且移动到上级目录(/usr/local)
4.解压缩mysql-connector-java-5.1.12.tar.gz文件,
并且复制文件mysql-connector-java-5.1.12-bin.jar到/usr/local/sqoop-1.4.7.bin__hadoop-2.6.0/lib中。
5.配置环境变量
cd /usr/local/sqoop-1.4.7.bin__hadoop-2.6.0/conf
cp sqoop-env-template.sh sqoop-env.sh
vim sqoop-env.sh
根据情况添加配置信息 内容如下:
# Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # included in all the hadoop scripts with source command # should not be executable directly # also should not be passed any arguments, since we need original $* # Set Hadoop-specific environment variables here. #Set path to where bin/hadoop is available export HADOOP_COMMON_HOME=/usr/local/hadoop-2.6.5 #Set path to where hadoop-*-core.jar is available export HADOOP_MAPRED_HOME=/usr/local/hadoop-2.6.5 #set the path to where bin/hbase is available export HBASE_HOME=/usr/local/hbase-1.4.8 #Set the path to where bin/hive is available export HIVE_HOME=/usr/local/apache-hive-2.1.0-bin #Set the path for where zookeper config dir is export ZOOCFGDIR=/usr/local/zookeeper/conf
6.配置linux的环境变量
cd
vim .bash_profile
在结尾添加如下代码:
#sqoop export SQOOP_HOME=/usr/local/sqoop-1.4.7.bin__hadoop-2.6.0/ export PATH=$PATH:$SQOOP_HOME/bin
7.保存退出;时文件生效:
source .bash_profile
8.测试与mysql的连接:
cd /usr/local/sqoop-1.4.7.bin__hadoop-2.6.0/bin
sqoop list-databases --connect jdbc:mysql://localhost:3306 --username root -P
输入密码后会显示具体的数据库列表
9.查看具体数据库中的 表信息:
sqoop list-tables --connect jdbc:mysql://localhost:3306/db1 --username root -P
10.执行查询语句:
sqoop -eval --connect jdbc:mysql://localhost:3306/db1 --username root -P --query "select * from a limit 5"
相关文章
- 关于用 ZIP 包 安装 MySql 的详细步骤以及可能的错误
- SQOOP安装部署
- Win10 环境安装tesseract-ocr 4.00并配置环境变量
- 在CentOS 7上安装Python3
- Angularjs学习---ubuntu12.04中karma安装配置中常见的问题总结
- GitLab - 安装并启动GitLab
- centos7.x 64位 rpm安装JDK8
- Hadoop sqoop数据导入导出工具安装
- CentOS下MySQL安装后配置和设置
- DayDayUp:平均每篇文章1毛! 本博主自2020年6月1日起,如有任何问题可在博客贴吧留言或者私信博主(包括并不限于GUI软件编写、安装及编程语言中的bug、AI算法设计等),非诚勿扰!
- Python:Python语言编程软件安装的几大姿势之详细攻略
- 已解决pip install lxml正确安装lxml模块
- Docker 安装 vsftpd
- linux安装-版本选择-终极决定
- IDAPython安装
- Linux 软件安装到 /usr,/usr/local/ 还是 /opt 目录?
- CUDA+torch+vscode安装教程(python3.8)
- CentOS7安装CDH(十):CDH中安装Spark2
- IP-guard数据防泄密软件如何安装客户端和日常使用
- 【大数据开发运维解决方案】Hadoop+Mysql+Hive+zookeeper+kafka+Hbase+Sqoop+Kylin单机伪分布式安装及官方案例详细文档
- CentOS 7 安装 Gitlab