304      财经资讯

数据预处理__使用手册(new)_机器学习-阿里云

目录

• 加权采样

• 随机采样

• 过滤与映射

• 分层采样

• join

• 合并列

• UNION

• 增加序号列

• 拆分

• 缺失值填充

• 归一化

• 标准化

• 类型转换

加权采样

以加权方式生成采样数据；权重列必须为double或int类型，按照该列的value大小采样；如col的值是1.2和1.0；则value=1.2所属样本的被采样的概率就大一些。

参数设置

参数框

• 可手动输入采样个数（或者采样比例）
• 可以选择放回采样或者不放回采样，默认为不放回，勾选后变为放回
• 下拉框选择加权列，加权列支持double型和bigint型
• 可配置随机数种子，默认系统自动分配

PAI 命令

PAI -name WeightedSample -project algo_public -DprobCol="previous" -DsampleSize="500" 
    -DoutputTableName="test2" -DinputPartitions="pt=20150501" -DinputTableName="bank_data_partition";

算法参数

注：当sampleSize 与 sampleRatio都为空时，报错； 当sampleSize 与 sampleRatio都不为空时，以sampleSize为准。

随机采样

以随机方式生成采样数据，每次采样是各自独立的。

参数设置

• 可手动输入采样个数（或者采样比例）
• 可以选择放回采样或者不放回采样，默认为不放回，勾选后变为放回
• 可配置随机数种子，默认系统自动分配
• 可以自己配置并发计算核心数目与内存，默认系统自动分配

PAI 命令

pai -name RandomSample -project algo_public 
    -DinputTableName=wbpc 
    -DoutputTableName=wpbc_sample 
    -DsampleSize=100 
    -Dreplace=false 
    -DrandomSeed=1007;

算法参数

注：当sampleSize 与 sampleRatio都为空时，报错； 当sampleSize 与 sampleRatio都不为空时，以sampleSize为准。

过滤与映射

对数据按照过滤表达式进行筛选；可以重命名字段名；

参数设置

1、通过where条件实现数据过滤，与SQL类似，如：

- 筛选条件：目前操作符支持”=”，”!=”，”>”，”<”， “>=”和“<=”，like，rlike    

2、重命名字段

PAI命令

PAI -name Filter -project algo_public -DoutTableName="test_9" -DinputPartitions="pt=20150501"
    -DinputTableName="bank_data_partition" -Dfilter="age>=40";

• name: 组件名字
• project: project名字，用于指定算法所在空间。系统默认是algo_public，用户自己更改后系统会报错
• outTableName: 输出表的名字
• inputPartitions：（可选）训练输入表分区。输入表对应的输入分区，选中全表则为None
• inputTableName: 输入表的名字
• filter: where筛选条件，目前操作符支持”=”，”!=”，”>”，”<”， “>=”，“<=”,“like”和“rlike”

分层采样

数据集分层抽取一定比例或者一定数据的随机样本

参数设置

参数框

• 下拉框选择分组列（最大支持100个分组）
• 可手动输入分组的采样个数（或者采样比例）
• 可配置随机数种子，默认1234567
• 可以自己配置并发计算核心数目与内存，默认系统自动分配

pai 命令

PAI -name StratifiedSample -project algo_public 
    -DinputTableName="test_input" 
    -DoutputTableName="test_output" 
    -DstrataColName="label" 
    -DsampleSize="A:200,B:300,C:500" 
    -DrandomSeed=1007 
    -Dlifecycle=30

算法参数

注：当sampleSize 与 sampleRatio都为空时，报错； 当sampleSize 与 sampleRatio都不为空时，以sampleSize为准。

示例

源数据

1. 创建实验

2. 选取分组列

• 选择play列，做为分组列

3. 配置采样个数

• play=Yes的分组采4条；play=No有分组采3条

4. 采样结果

join

两张表通过关联信息，合成一张表，并决定输出的字段;与SQL的join语句功能类似

参数设置

填写参数

• 连接类型支持：左连接，内连接，右连接，全连接
• 关联条件目前只支持等式
• 可手动添加或删除关联条件

PAI 命令

不提供pai命令

合并列

将两张表的数据按列合并，需要表的行数保持一致

参数设置

如图：左表选择输入列右表选择输入列

• 选择的两张表行数需保持一致

• 左表与右表选择的输出列名不能重复

• 选择输出字段列时可手动更改输出字段名

• 若左表右表均不选择输出列，则默认输出全表。此时勾选‘是否自动重命名输出列’，会将重复列重命名后输出

PAI 命令

PAI -name AppendColumns -project algo_public -DoutputTableColNames="petal_length,petal_width,petal_length2,petal_width2"
    -DautoRenameCol="false" -DoutputTableName="pai_temp_770_6840_1" -DinputTableNames="iris_twopartition,iris_twopartition"
    -DinputPartitionsInfoList="dt=20150125/dp=20150124;dt=20150124/dp=20150123" 
    -DselectedColNamesList="petal_length,petal_width;sepal_length,sepal_width";

• name: 组件名字
• project: project名字，用于指定算法所在空间。系统默认是algo_public，用户自己更改后系统会报错
• outputTableColNames: 新表中各列的列名。逗号分隔，如果autoRenameCol为true，则此参数无效
• autoRenameCol: （可选）输出表是否自动重命名列，true为重命名，false不进行重命名，默认false
• outputTableName: 输出的表名
• inputTableNames：输入的表名，如果有多个，逗号分隔
• inputPartitionsInfoList：（可选）输入表对应选择的partition列表，同一个表的各partition按逗号分隔，不同表的partition分号分隔
• selectedColNamesList：选择输入的列名，同张表之间列名用逗号分隔，不同表之间用分号分隔

UNION

将两张表的数据按行合并，左表及右表选择输出的字段个数以及类型应保持一致。整合了union和union all的功能。

参数设置

调整参数，如下：

• 进行联合操作时，左右表选择的列数需相同，对应列的类型需保证一致
• 可根据实际情况在条件框中手动输入已选字段的过滤条件，默认情况下全表，目前操作符支持”=”，”!=”，”>”，”<”， “>=”，“<=”，“like” 和 “rlike”
• 系统默认勾选去重。勾选后将会对生成的数据表的重复行进行去重操作（distinct）左表联合列右表联合列

PAI 命令

不提供pai命令

增加序号列

在数据表第一列追加ID列，并将append id后的内容存为新表

参数设置

略

PAI命令

PAI -name AppendId -project algo_public -DIDColName="append_id" -DoutputTableName="test_11" -DinputTableName="bank_data" 
    -DselectedColNames="age,campaign,cons_conf_idx,cons_price_idx,emp_var_rate,euribor3m,nr_employed,pdays,poutcome,previous,y";

• name: 组件名字
• project: project名字，用于指定算法所在空间。系统默认是algo_public，用户自己更改后系统会报错
• IDColName：追加的ID列列名，从0开始编号，依次为0,1,2,3…
• outputTableNames: 输出表的名字
• inputTableNmae: 输入表的名字
• selectedColNames: 要保留的字段名，多个用逗号分隔

拆分

背景

• 对输入表或分区进行按比例拆分，分别写入两张输出表。

算法组件

• 拆分组件，对应两个输出桩
• 如图所示配置0.6，则左边的输出桩对应60%的数据，右边对应40%的数据

PAI命令

pai -name split -project algo_public 
    -DinputTableName=wbpc 
    -Doutput1TableName=wpbc_split1 
    -Doutput2TableName=wpbc_split2 
    -Dfraction=0.25;

参数设置

缺失值填充

缺失值填充用来将空值或者一个指定的值替换为最大值，最小值，均值或者一个自定义的值。可以通过给定一个缺失值的配置列表，来实现将输入表的缺失值用指定的值来填充。

• 可以将数值型的空值替换为最大值，最小值，均值或者一个自定义的值
• 可以将字符型的空值，空字符串，空值和空字符串，指定值替换为一个自定义的值
• 待填充的缺失值可以选择空值或空字符，也可以自定义
• 缺失值若选择空字符，则填充的目标列应是string型
• 数值型替换可以自定义，也可以直接选择替换成数值最大值，最小值或者均值

缺失值填充界面

两个输入桩依次对应参数为：

• inputTableName 输入表，即要填充的表

• inputParaTableName 配置输入表，即缺失值填充节点生成的参数列表，通过此参数，可以将一张表的配置参数应用到一张新的表

两个输出桩依次对应参数为：

• outputTableName 输出表，即填充完成的表

• outputParaTableName 输出配置表，用于应用到其他的数据集上

缺失值填充参数界面

• 填充的字段，原值，替换为三个部分组成了config参数，分别对应config参数的三个部分：列名，原值，替换值

PAI 命令

PAI -name FillMissingValues -project algo_public -Dconfigs="poutcome,null-empty,testing" 
    -DoutputTableName="test_3" -DinputPartitions="pt=20150501" -DinputTableName="bank_data_partition";

算法参数

实例

测试数据

新建数据SQL

drop table if exists fill_missing_values_test_input;
create table fill_missing_values_test_input(
    col_string string,
    col_bigint bigint,
    col_double double,
    col_boolean boolean,
    col_datetime datetime);
insert overwrite table fill_missing_values_test_input
select
    *
from
(
    select
        '01' as col_string,
        10 as col_bigint,
        10.1 as col_double,
        True as col_boolean,
        cast('2016-07-01 10:00:00' as datetime) as col_datetime
    from dual
    union all
        select
            cast(null as string) as col_string,
            11 as col_bigint,
            10.2 as col_double,
            False as col_boolean,
            cast('2016-07-02 10:00:00' as datetime) as col_datetime
        from dual
    union all
        select
            '02' as col_string,
            cast(null as bigint) as col_bigint,
            10.3 as col_double,
            True as col_boolean,
            cast('2016-07-03 10:00:00' as datetime) as col_datetime
        from dual
    union all
        select
            '03' as col_string,
            12 as col_bigint,
            cast(null as double) as col_double,
            False as col_boolean,
            cast('2016-07-04 10:00:00' as datetime) as col_datetime
        from dual
    union all
        select
            '04' as col_string,
            13 as col_bigint,
            10.4 as col_double,
            cast(null as boolean) as col_boolean,
            cast('2016-07-05 10:00:00' as datetime) as col_datetime
        from dual
    union all
        select
            '05' as col_string,
            14 as col_bigint,
            10.5 as col_double,
            True as col_boolean,
            cast(null as datetime) as col_datetime
        from dual
) tmp;

输入数据说明

+------------+------------+------------+-------------+--------------+
| col_string | col_bigint | col_double | col_boolean | col_datetime |
+------------+------------+------------+-------------+--------------+
| 04         | 13         | 10.4       | NULL        | 2016-07-05 10:00:00 |
| 02         | NULL       | 10.3       | true        | 2016-07-03 10:00:00 |
| 03         | 12         | NULL       | false       | 2016-07-04 10:00:00 |
| NULL       | 11         | 10.2       | false       | 2016-07-02 10:00:00 |
| 01         | 10         | 10.1       | true        | 2016-07-01 10:00:00 |
| 05         | 14         | 10.5       | true        | NULL         |
+------------+------------+------------+-------------+--------------+

运行命令

drop table if exists fill_missing_values_test_input_output;
drop table if exists fill_missing_values_test_input_model_output;
PAI -name FillMissingValues
-project algo_public
-Dconfigs="col_double,null,mean;col_string,null-empty,str_type_empty;col_bigint,null,max;col_boolean,null,true;col_datetime,null,2016-07-06 10:00:00"
-DoutputParaTableName="fill_missing_values_test_input_model_output"
-Dlifecycle="28"
-DoutputTableName="fill_missing_values_test_input_output"
-DinputTableName="fill_missing_values_test_input";
drop table if exists fill_missing_values_test_input_output_using_model;
drop table if exists fill_missing_values_test_input_output_using_model_model_output;
PAI -name FillMissingValues
-project algo_public
-DoutputParaTableName="fill_missing_values_test_input_output_using_model_model_output"
-DinputParaTableName="fill_missing_values_test_input_model_output"
-Dlifecycle="28"
-DoutputTableName="fill_missing_values_test_input_output_using_model"
-DinputTableName="fill_missing_values_test_input";

运行结果

+------------+------------+------------+-------------+--------------+
| col_string | col_bigint | col_double | col_boolean | col_datetime |
+------------+------------+------------+-------------+--------------+
| 04         | 13         | 10.4       | true        | 2016-07-05 10:00:00 |
| 02         | 14         | 10.3       | true        | 2016-07-03 10:00:00 |
| 03         | 12         | 10.3       | false       | 2016-07-04 10:00:00 |
| str_type_empty | 11         | 10.2       | false       | 2016-07-02 10:00:00 |
| 01         | 10         | 10.1       | true        | 2016-07-01 10:00:00 |
| 05         | 14         | 10.5       | true        | 2016-07-06 10:00:00 |
+------------+------------+------------+-------------+--------------+

+------------+------------+
| feature    | json       |
+------------+------------+
| col_string | {"name": "fillMissingValues", "type": "string", "paras":{"missing_value_type": "null-empty",  "replaced_value": "str_type_empty"}} |
| col_bigint | {"name": "fillMissingValues", "type": "bigint", "paras":{"missing_value_type": "null",  "replaced_value": 14}} |
| col_double | {"name": "fillMissingValues", "type": "double", "paras":{"missing_value_type": "null",  "replaced_value": 10.3}} |
| col_boolean | {"name": "fillMissingValues", "type": "boolean", "paras":{"missing_value_type": "null",  "replaced_value": 1}} |
| col_datetime | {"name": "fillMissingValues", "type": "datetime", "paras":{"missing_value_type": "null",  "replaced_value": 1467770400000}} |
+------------+------------+

+------------+------------+------------+-------------+--------------+
| col_string | col_bigint | col_double | col_boolean | col_datetime |
+------------+------------+------------+-------------+--------------+
| 04         | 13         | 10.4       | true        | 2016-07-05 10:00:00 |
| 02         | 14         | 10.3       | true        | 2016-07-03 10:00:00 |
| 03         | 12         | 10.3       | false       | 2016-07-04 10:00:00 |
| str_type_empty | 11         | 10.2       | false       | 2016-07-02 10:00:00 |
| 01         | 10         | 10.1       | true        | 2016-07-01 10:00:00 |
| 05         | 14         | 10.5       | true        | 2016-07-06 10:00:00 |
+------------+------------+------------+-------------+--------------+

+------------+------------+
| feature    | json       |
+------------+------------+
| col_string | {"name": "fillMissingValues", "type": "string", "paras":{"missing_value_type": "null-empty",  "replaced_value": "str_type_empty"}} |
| col_bigint | {"name": "fillMissingValues", "type": "bigint", "paras":{"missing_value_type": "null",  "replaced_value": 14}} |
| col_double | {"name": "fillMissingValues", "type": "double", "paras":{"missing_value_type": "null",  "replaced_value": 10.3}} |
| col_boolean | {"name": "fillMissingValues", "type": "boolean", "paras":{"missing_value_type": "null",  "replaced_value": 1}} |
| col_datetime | {"name": "fillMissingValues", "type": "datetime", "paras":{"missing_value_type": "null",  "replaced_value": 1467770400000}} |
+------------+------------+

PAI -name Normalize -project algo_public -DkeepOriginal="true" -DoutputTableName="test_4" -DinputPartitions="pt=20150501" 
    -DinputTableName="bank_data_partition" -DselectedColNames="emp_var_rate,euribor3m";

drop table if exists normalize_test_input;
create table normalize_test_input(
    col_string string,
    col_bigint bigint,
    col_double double,
    col_boolean boolean,
    col_datetime datetime);
insert overwrite table normalize_test_input 
select 
    *
from 
(
    select
        '01' as col_string,
        10 as col_bigint,
        10.1 as col_double,
        True as col_boolean,
        cast('2016-07-01 10:00:00' as datetime) as col_datetime
    from dual
    union all
        select
            cast(null as string) as col_string,
            11 as col_bigint,
            10.2 as col_double,
            False as col_boolean,
            cast('2016-07-02 10:00:00' as datetime) as col_datetime
        from dual
    union all
        select
            '02' as col_string,
            cast(null as bigint) as col_bigint,
            10.3 as col_double,
            True as col_boolean,
            cast('2016-07-03 10:00:00' as datetime) as col_datetime
        from dual
    union all
        select
            '03' as col_string,
            12 as col_bigint,
            cast(null as double) as col_double,
            False as col_boolean,
            cast('2016-07-04 10:00:00' as datetime) as col_datetime
        from dual
    union all
        select
            '04' as col_string,
            13 as col_bigint,
            10.4 as col_double,
            cast(null as boolean) as col_boolean,
            cast('2016-07-05 10:00:00' as datetime) as col_datetime
        from dual
    union all
        select
            '05' as col_string,
            14 as col_bigint,
            10.5 as col_double,
            True as col_boolean,
            cast(null as datetime) as col_datetime
        from dual
) tmp;

+------------+------------+------------+-------------+--------------+
| col_string | col_bigint | col_double | col_boolean | col_datetime |
+------------+------------+------------+-------------+--------------+
| 01         | 10         | 10.1       | true        | 2016-07-01 10:00:00 |
| NULL       | 11         | 10.2       | false       | 2016-07-02 10:00:00 |
| 02         | NULL       | 10.3       | true        | 2016-07-03 10:00:00 |
| 03         | 12         | NULL       | false       | 2016-07-04 10:00:00 |
| 04         | 13         | 10.4       | NULL        | 2016-07-05 10:00:00 |
| 05         | 14         | 10.5       | true        | NULL         |
+------------+------------+------------+-------------+--------------+

drop table if exists normalize_test_input_output;
drop table if exists normalize_test_input_model_output;
PAI -name Normalize
-project algo_public
-DoutputParaTableName="normalize_test_input_model_output"
-Dlifecycle="28"
-DoutputTableName="normalize_test_input_output"
-DinputTableName="normalize_test_input"
-DselectedColNames="col_double,col_bigint"
-DkeepOriginal="true";
drop table if exists normalize_test_input_output_using_model;
drop table if exists normalize_test_input_output_using_model_model_output;
PAI -name Normalize
-project algo_public
-DoutputParaTableName="normalize_test_input_output_using_model_model_output"
-DinputParaTableName="normalize_test_input_model_output"
-Dlifecycle="28"
-DoutputTableName="normalize_test_input_output_using_model"
-DinputTableName="normalize_test_input";

+------------+------------+------------+-------------+--------------+-----------------------+-----------------------+
| col_string | col_bigint | col_double | col_boolean | col_datetime | normalized_col_bigint | normalized_col_double |
+------------+------------+------------+-------------+--------------+-----------------------+-----------------------+
| 01         | 10         | 10.1       | true        | 2016-07-01 10:00:00 | 0.0                   | 0.0                   |
| NULL       | 11         | 10.2       | false       | 2016-07-02 10:00:00 | 0.25                  | 0.2499999999999989    |
| 02         | NULL       | 10.3       | true        | 2016-07-03 10:00:00 | NULL                  | 0.5000000000000022    |
| 03         | 12         | NULL       | false       | 2016-07-04 10:00:00 | 0.5                   | NULL                  |
| 04         | 13         | 10.4       | NULL        | 2016-07-05 10:00:00 | 0.75                  | 0.7500000000000011    |
| 05         | 14         | 10.5       | true        | NULL         | 1.0                   | 1.0                   |
+------------+------------+------------+-------------+--------------+-----------------------+-----------------------+

+------------+------------+
| feature    | json       |
+------------+------------+
| col_bigint | {"name": "normalize", "type":"bigint", "paras":{"min":10, "max": 14}} |
| col_double | {"name": "normalize", "type":"double", "paras":{"min":10.1, "max": 10.5}} |
+------------+------------+

+------------+------------+------------+-------------+--------------+
| col_string | col_bigint | col_double | col_boolean | col_datetime |
+------------+------------+------------+-------------+--------------+
| 01         | 0.0        | 0.0        | true        | 2016-07-01 10:00:00 |
| NULL       | 0.25       | 0.2499999999999989 | false       | 2016-07-02 10:00:00 |
| 02         | NULL       | 0.5000000000000022 | true        | 2016-07-03 10:00:00 |
| 03         | 0.5        | NULL       | false       | 2016-07-04 10:00:00 |
| 04         | 0.75       | 0.7500000000000011 | NULL        | 2016-07-05 10:00:00 |
| 05         | 1.0        | 1.0        | true        | NULL         |
+------------+------------+------------+-------------+--------------+

+------------+------------+
| feature    | json       |
+------------+------------+
| col_bigint | {"name": "normalize", "type":"bigint", "paras":{"min":10, "max": 14}} |
| col_double | {"name": "normalize", "type":"double", "paras":{"min":10.1, "max": 10.5}} |
+------------+------------+

PAI -name Standardize -project algo_public -DkeepOriginal="false" -DoutputTableName="test_5" 
    -DinputPartitions="pt=20150501" -DinputTableName="bank_data_partition" -DselectedColNames="euribor3m,pdays";

drop table if exists standardize_test_input;
create table standardize_test_input(
    col_string string,
    col_bigint bigint,
    col_double double,
    col_boolean boolean,
    col_datetime datetime);
insert overwrite table standardize_test_input 
select 
    *
from 
(
    select
        '01' as col_string,
        10 as col_bigint,
        10.1 as col_double,
        True as col_boolean,
        cast('2016-07-01 10:00:00' as datetime) as col_datetime
    from dual
    union all
        select
            cast(null as string) as col_string,
            11 as col_bigint,
            10.2 as col_double,
            False as col_boolean,
            cast('2016-07-02 10:00:00' as datetime) as col_datetime
        from dual
    union all
        select
            '02' as col_string,
            cast(null as bigint) as col_bigint,
            10.3 as col_double,
            True as col_boolean,
            cast('2016-07-03 10:00:00' as datetime) as col_datetime
        from dual
    union all
        select
            '03' as col_string,
            12 as col_bigint,
            cast(null as double) as col_double,
            False as col_boolean,
            cast('2016-07-04 10:00:00' as datetime) as col_datetime
        from dual
    union all
        select
            '04' as col_string,
            13 as col_bigint,
            10.4 as col_double,
            cast(null as boolean) as col_boolean,
            cast('2016-07-05 10:00:00' as datetime) as col_datetime
        from dual
    union all
        select
            '05' as col_string,
            14 as col_bigint,
            10.5 as col_double,
            True as col_boolean,
            cast(null as datetime) as col_datetime
        from dual
) tmp;

+------------+------------+------------+-------------+--------------+
| col_string | col_bigint | col_double | col_boolean | col_datetime |
+------------+------------+------------+-------------+--------------+
| 01         | 10         | 10.1       | true        | 2016-07-01 10:00:00 |
| NULL       | 11         | 10.2       | false       | 2016-07-02 10:00:00 |
| 02         | NULL       | 10.3       | true        | 2016-07-03 10:00:00 |
| 03         | 12         | NULL       | false       | 2016-07-04 10:00:00 |
| 04         | 13         | 10.4       | NULL        | 2016-07-05 10:00:00 |
| 05         | 14         | 10.5       | true        | NULL         |
+------------+------------+------------+-------------+--------------+

drop table if exists standardize_test_input_output;
drop table if exists standardize_test_input_model_output;
PAI -name Standardize
-project algo_public
-DoutputParaTableName="standardize_test_input_model_output"
-Dlifecycle="28"
-DoutputTableName="standardize_test_input_output"
-DinputTableName="standardize_test_input"
-DselectedColNames="col_double,col_bigint"
-DkeepOriginal="true";
drop table if exists standardize_test_input_output_using_model;
drop table if exists standardize_test_input_output_using_model_model_output;
PAI -name Standardize
-project algo_public
-DoutputParaTableName="standardize_test_input_output_using_model_model_output"
-DinputParaTableName="standardize_test_input_model_output"
-Dlifecycle="28"
-DoutputTableName="standardize_test_input_output_using_model"
-DinputTableName="standardize_test_input";

+------------+------------+------------+-------------+--------------+--------------------+--------------------+
| col_string | col_bigint | col_double | col_boolean | col_datetime | stdized_col_bigint | stdized_col_double |
+------------+------------+------------+-------------+--------------+--------------------+--------------------+
| 01         | 10         | 10.1       | true        | 2016-07-01 10:00:00 | -1.2649110640673518 | -1.2649110640683832 |
| NULL       | 11         | 10.2       | false       | 2016-07-02 10:00:00 | -0.6324555320336759 | -0.6324555320341972 |
| 02         | NULL       | 10.3       | true        | 2016-07-03 10:00:00 | NULL               | 0.0                |
| 03         | 12         | NULL       | false       | 2016-07-04 10:00:00 | 0.0                | NULL               |
| 04         | 13         | 10.4       | NULL        | 2016-07-05 10:00:00 | 0.6324555320336759 | 0.6324555320341859 |
| 05         | 14         | 10.5       | true        | NULL         | 1.2649110640673518 | 1.2649110640683718 |
+------------+------------+------------+-------------+--------------+--------------------+--------------------+

+------------+------------+
| feature    | json       |
+------------+------------+
| col_bigint | {"name": "standardize", "type":"bigint", "paras":{"mean":12, "std": 1.58113883008419}} |
| col_double | {"name": "standardize", "type":"double", "paras":{"mean":10.3, "std": 0.1581138830082909}} |
+------------+------------+

+------------+------------+------------+-------------+--------------+
| col_string | col_bigint | col_double | col_boolean | col_datetime |
+------------+------------+------------+-------------+--------------+
| 01         | -1.2649110640673515 | -1.264911064068383 | true        | 2016-07-01 10:00:00 |
| NULL       | -0.6324555320336758 | -0.6324555320341971 | false       | 2016-07-02 10:00:00 |
| 02         | NULL       | 0.0        | true        | 2016-07-03 10:00:00 |
| 03         | 0.0        | NULL       | false       | 2016-07-04 10:00:00 |
| 04         | 0.6324555320336758 | 0.6324555320341858 | NULL        | 2016-07-05 10:00:00 |
| 05         | 1.2649110640673515 | 1.2649110640683716 | true        | NULL         |
+------------+------------+------------+-------------+--------------+

+------------+------------+
| feature    | json       |
+------------+------------+
| col_bigint | {"name": "standardize", "type":"bigint", "paras":{"mean":12, "std": 1.58113883008419}} |
| col_double | {"name": "standardize", "type":"double", "paras":{"mean":10.3, "std": 0.1581138830082909}} |
+------------+------------+

PAI -name type_transform 
    -project algo_public 
    -DinputTable="pai_dense"
    -DselectedCols="gail,loss,work_year" 
    -Dpre_type="double"
    -Dnew_type="bigint" 
    -DoutputTable="pai_temp_2250_20272_1"
    -Dlifecycle="28"