读取HDFS数据写入MySQL_大数据培训
读取HDFS数据写入MySQL
1）将上个案例上传的文件改名
[atguigu@hadoop102 datax]$ hadoop fs -mv /student.txt* /student.txt
2）查看官方模板
[atguigu@hadoop102 datax]$ python bin/datax.py -r hdfsreader -w mysqlwriter
{
“job”: {
“content”: [
{
“reader”: {
“name”: “hdfsreader”,
“parameter”: {
“column”: [],
“defaultFS”: “”,
“encoding”: “UTF-8”,
“fieldDelimiter”: “,”,
“fileType”: “orc”,
“path”: “”
}
},
“writer”: {
“name”: “mysqlwriter”,
“parameter”: {
“column”: [],
“connection”: [
{
“jdbcUrl”: “”,
“table”: []
}
],
“password”: “”,
“preSql”: [],
“session”: [],
“username”: “”,
“writeMode”: “”
}
}
}
],
“setting”: {
“speed”: {
“channel”: “”
}
}
}
}
3）创建配置文件
[atguigu@hadoop102 datax]$ vim job/hdfs2mysql.json
{
“job”: {
“content”: [
{
“reader”: {
“name”: “hdfsreader”,
“parameter”: {
“column”: [“*”],
“defaultFS”: “hdfs://hadoop102:9000”,
“encoding”: “UTF-8”,
“fieldDelimiter”: “\t”,
“fileType”: “text”,
“path”: “/student.txt”
}
},
“writer”: {
“name”: “mysqlwriter”,
“parameter”: {
“column”: [
“id”,
“name”
],
“connection”: [
{
“jdbcUrl”: “jdbc:mysql://hadoop102:3306/datax”,
“table”: [“student2”]
}
],
“password”: “000000”,
“username”: “root”,
“writeMode”: “insert”
}
}
}
],
“setting”: {
“speed”: {
“channel”: “1”
}
}
}
}
4）在MySQL的datax数据库中创建student2
mysql> use datax;
mysql> create table student2(id int,name varchar(20));
5）执行任务
6）查看student2表
