benchmarksql安装部署

benchmarksql 在5.0版本去除了对MySQL的TPC-C测试支持，本文通过修改部分源码，让benchmarksql 5.0支持MySQL。
1. 环境:

    CentOS 7.2
    benchmarksql 5.0
    Percona MySQL Server 5.7.19

2. 下载编译benchmarksql源码

2.1 首先安装java开发环境，具体步骤略过，本文涉及到的操作在 java 1.8.0 环境下测试通过。

2.2 安装ant工具。
yum install ant

2.3 下载benchmarksql 5.0 源码，解压。
https://sourceforge.net/projects/benchmarksql/
cd benchmarksql-5.0

2.4 编译
ant

此时会编译出一个版本 benchmarksql-5.0/dist/BenchmarkSQL-5.0.jar，但是该版本并不支持MySQL的TPC-C测试，需要做如下的修改。
3 修改benchmarksql源码

3.1 修改benchmarksql-5.0/src/client/jTPCC.java，增加mysql相关部分，如下所示：
if (iDB.equals("firebird"))
        dbType = DB_FIREBIRD;
    else if (iDB.equals("oracle"))
        dbType = DB_ORACLE;
    else if (iDB.equals("postgres"))
        dbType = DB_POSTGRES;
    else if (iDB.equals("mysql"))
        dbType = DB_UNKNOWN;
    else
    {
        log.error("unknown database type '" + iDB + "'");
        return;
    }

3.2 修改benchmarksql-5.0/src/client/jTPCCConnection.java， SQL子查询增加"AS L"别名，如下所示：
default:
        stmtStockLevelSelectLow = dbConn.prepareStatement(
            "SELECT count(*) AS low_stock FROM (" +
            "    SELECT s_w_id, s_i_id, s_quantity " +
            "        FROM bmsql_stock " +
            "        WHERE s_w_id = ? AND s_quantity < ? AND s_i_id IN (" +
            "            SELECT ol_i_id " +
            "                FROM bmsql_district " +
            "                JOIN bmsql_order_line ON ol_w_id = d_w_id " +
            "                 AND ol_d_id = d_id " +
            "                 AND ol_o_id >= d_next_o_id - 20 " +
            "                 AND ol_o_id < d_next_o_id " +
            "                WHERE d_w_id = ? AND d_id = ? " +
            "        ) " +
            "    ) AS L");
        break;

3.3 编译修改后的源码，此时得到的benchmarksql版本 benchmarksql-5.0/dist/BenchmarkSQL-5.0.jar 已经支持MySQL的TPC-C测试。

cd benchmarksql-5.0
ant

4 修改相关脚本，支持MySQL

4.1. 在benchmarksql-5.0/run目录下，创建文件props.mysql，内容如下：
db=mysql
driver=com.mysql.jdbc.Driver
conn=jdbc:mysql://127.0.0.1:3306/db_admin
user=root
password=123456
warehouses=1
loadWorkers=4
terminals=1
//To run specified transactions per terminal- runMins must equal zero
runTxnsPerTerminal=10
//To run for specified minutes- runTxnsPerTerminal must equal zero
runMins=0
//Number of total transactions per minute
limitTxnsPerMin=300
//Set to true to run in 4.x compatible mode. Set to false to use the
//entire configured database evenly.
terminalWarehouseFixed=true
//The following five values must add up to 100
//The default percentages of 45, 43, 4, 4 & 4 match the TPC-C spec
newOrderWeight=45
paymentWeight=43
orderStatusWeight=4
deliveryWeight=4
stockLevelWeight=4
// Directory name to create for collecting detailed result data.
// Comment this out to suppress.
resultDirectory=my_result_%tY-%tm-%td_%tH%tM%tS
osCollectorScript=./misc/os_collector_linux.py
osCollectorInterval=1
//osCollectorSSHAddr=user@dbhost
osCollectorDevices=net_eth0 blk_sda

4.2. 修改 文件：benchmarksql-5.0/run/funcs.sh，添加mysql 数据库类型。
function setCP()
{
    case "$(getProp db)" in
    firebird)
        cp="../lib/firebird/*:../lib/*"
        ;;
    oracle)
        cp="../lib/oracle/*"
        if [ ! -z "${ORACLE_HOME}" -a -d ${ORACLE_HOME}/lib ] ; then
        cp="${cp}:${ORACLE_HOME}/lib/*"
        fi
        cp="${cp}:../lib/*"
        ;;
    postgres)
        cp="../lib/postgres/*:../lib/*"
        ;;
    mysql)
        cp="../lib/mysql/*:../lib/*"
        ;;
    esac
    myCP=".:${cp}:../dist/*"
    export myCP
}

...省略

case "$(getProp db)" in
    firebird|oracle|postgres|mysql)
    ;;
    "") echo "ERROR: missing db= config option in ${PROPS}" >&2
    exit 1
    ;;
    *)  echo "ERROR: unsupported database type 'db=$(getProp db)' in ${PROPS}" >&2
    exit 1
    ;;
esac

4.3 添加mysql java connector驱动，mysql-connector-java-5.1.45.jar 需自行下载。

mkdir -p benchmarksql-5.0/lib/mysql
cp mysql-connector-java-5.1.45.jar benchmarksql-5.0/lib/mysql/

4.4 修改benchmarksql-5.0/run/runDatabaseBuild.sh，去掉extraHistID
AFTER_LOAD="indexCreates foreignKeys extraHistID buildFinish"
修改为：
AFTER_LOAD="indexCreates foreignKeys buildFinish"
5 测试MySQL TPC-C

cd benchmarksql-5.0/run
./runDatabaseBuild.sh props.mysql
./runBenchmark.sh props.mysql

修改后的benchmarksql对MySQL进行TPC-C测试，结果如下所示：
02:58:47,071 [Thread-1] INFO   jTPCC : Term-00,
02:58:47,071 [Thread-1] INFO   jTPCC : Term-00, Measured tpmC (NewOrders) = 136.71
02:58:47,071 [Thread-1] INFO   jTPCC : Term-00, Measured tpmTOTAL = 298.81
02:58:47,071 [Thread-1] INFO   jTPCC : Term-00, Session Start     = 2018-12-27 02:53:46
02:58:47,071 [Thread-1] INFO   jTPCC : Term-00, Session End       = 2018-12-27 02:58:47
02:58:47,071 [Thread-1] INFO   jTPCC : Term-00, Transaction Count = 1494

修改props.mysql参数后重新执行,比如调整如下两个参数:

runTxnsPerTerminal=0
//To run for specified minutes- runTxnsPerTerminal must equal zero
runMins=20

重新执行:

./runDatabaseDestroy.sh my_postgres.properties
./runDatabaseBuild.sh my_postgres.properties
./runBenchmark.sh my_postgres.properties

6.输出报表

安装R

yum -y install R

执行完成后runBenchmark.sh后会在run目录下生成一个结果目录，执行如下的命令可以生成报表

./generateReport.sh my_result_2019-12-25_101117

 将如下html文件下载到本地即可查看

流量指标(Throughput，简称tpmC)
按照TPC的定义，流量指标描述了系统在执行Payment、Order-status、Delivery、Stock-Level这四种交易的同时，每分钟可以处理多少个New-Order交易。所有交易的响应时间必须满足TPC-C测试规范的要求。
流量指标值越大越好！

性价比(Price/Performance，简称Price/tpmC)
即测试系统价格（指在美国的报价）与流量指标的比值。
性价比越小越好！