2 September 2020,01:48:45 prefect.CloudFlowRunner INFO Beginning Flow run for 'store-etl' 2 September 2020,01:48:45 prefect.CloudFlowRunner INFO Starting flow run. 2 September 2020,01:48:45 prefect.CloudTaskRunner INFO Task 'date_hour_path': Starting task run... 2 September 2020,01:48:46 prefect.CloudTaskRunner INFO Task 'date_hour_path': finished task run for task with final state: 'Success' 2 September 2020,01:48:46 prefect.CloudTaskRunner INFO Task 'start': Starting task run... 2 September 2020,01:48:46 prefect.CloudTaskRunner INFO Task 'start': finished task run for task with final state: 'Success' 2 September 2020,01:48:46 prefect.CloudTaskRunner INFO Task 'get_S3_connection': Starting task run... 2 September 2020,01:48:47 prefect.CloudTaskRunner INFO Task 'get_S3_connection': finished task run for task with final state: 'Success' 2 September 2020,01:48:47 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:48:55 prefect.write_delta_lookup_table INFO file_list --> ['20200901-180618198', '20200901-180619579', '20200901-180621106', '20200901-180621386', '20200901-180622573', '20200901-180622707', '20200901-180622827', '20200901-180623029', '20200901-180623134', '20200901-180623267', '20200901-180623803', '20200901-180624062', '20200901-180624206', '20200901-180624370'] 2 September 2020,01:49:39 prefect.write_delta_lookup_table INFO Delta write complete in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-str-opstmt-hdr-typ/delta_raw/ . Table name - emd.str_opstmt_hdr_typ 2 September 2020,01:50:32 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:50:32 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:50:32 prefect.write_delta_lookup_table INFO file_list --> [] 2 September 2020,01:50:32 prefect.write_delta_lookup_table WARNING No transaction data in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-company/2020-09-01/18 2 September 2020,01:50:32 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:50:32 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:50:33 prefect.write_delta_lookup_table INFO file_list --> ['20200901-183615161', '20200901-183615639', '20200901-183616341', '20200901-183617086', '20200901-183617515', '20200901-183617853', '20200901-183618351', '20200901-183618827', '20200901-183619755', '20200901-183620039', '20200901-183620404', '20200901-183620587', '20200901-183620810', '20200901-183620974'] 2 September 2020,01:50:56 prefect.write_delta_lookup_table INFO Delta write complete in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-cmptr-typ/delta_raw/ . Table name - emd.cmptr_typ 2 September 2020,01:51:35 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:51:35 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:51:35 prefect.write_delta_lookup_table INFO file_list --> ['20200901-180617956', '20200901-180618232', '20200901-180619655', '20200901-180620931', '20200901-180621261', '20200901-180621470', '20200901-180621646', '20200901-180623042'] 2 September 2020,01:51:53 prefect.write_delta_lookup_table INFO Delta write complete in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-ck-coll-cntl/delta_raw/ . Table name - emd.ck_coll_cntl 2 September 2020,01:52:29 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:52:29 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:52:29 prefect.write_delta_lookup_table INFO file_list --> ['20200901-183619808', '20200901-183620076', '20200901-183620386', '20200901-183620643', '20200901-183620852', '20200901-183620990', '20200901-183621122', '20200901-183621277', '20200901-183621540', '20200901-183621995'] 2 September 2020,01:52:46 prefect.write_delta_lookup_table INFO Delta write complete in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-location-type/delta_raw/ . Table name - emd.location_typ 2 September 2020,01:53:17 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:53:17 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:53:17 prefect.write_delta_lookup_table INFO file_list --> [] 2 September 2020,01:53:17 prefect.write_delta_lookup_table WARNING No transaction data in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-retl-loc-segm/2020-09-01/18 2 September 2020,01:53:17 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:53:17 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:53:17 prefect.write_delta_lookup_table INFO file_list --> [] 2 September 2020,01:53:17 prefect.write_delta_lookup_table WARNING No transaction data in path - s3a://heb-stg-emd-store-raw-ora.pemd/firehose-emd-heb-dt/2020-09-01/18 2 September 2020,01:53:18 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:53:18 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:53:18 prefect.write_delta_lookup_table INFO file_list --> [] 2 September 2020,01:53:18 prefect.write_delta_lookup_table WARNING No transaction data in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-retl-unld/2020-09-01/18 2 September 2020,01:53:18 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:53:18 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:53:18 prefect.write_delta_lookup_table INFO file_list --> ['20200901-180017230', '20200901-180017718', '20200901-180018746', '20200901-180019392', '20200901-180019913', '20200901-180020335', '20200901-180020774', '20200901-180021084'] 2 September 2020,01:53:34 prefect.write_delta_lookup_table INFO Delta write complete in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-str-opstmt-typ/delta_raw/ . Table name - emd.str_opstmt_typ 2 September 2020,01:54:00 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:54:00 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:54:00 prefect.write_delta_lookup_table INFO file_list --> ['20200901-180016586', '20200901-180017367', '20200901-180017792', '20200901-180018709'] 2 September 2020,01:54:13 prefect.write_delta_lookup_table INFO Delta write complete in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-srs-sys/delta_raw/ . Table name - emd.srs_sys 2 September 2020,01:54:34 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:54:34 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:54:34 prefect.write_delta_lookup_table INFO file_list --> ['20200901-180015467', '20200901-180017182', '20200901-180017648', '20200901-180017924', '20200901-180018795', '20200901-180019659', '20200901-180020147', '20200901-180020749', '20200901-180020973', '20200901-180021275'] 2 September 2020,01:54:50 prefect.write_delta_lookup_table INFO Delta write complete in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-ecomm-mkt-area/delta_raw/ . Table name - emd.ecomm_mkt_area 2 September 2020,01:55:24 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:55:24 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:55:24 prefect.write_delta_lookup_table INFO file_list --> [] 2 September 2020,01:55:24 prefect.write_delta_lookup_table WARNING No transaction data in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-lin-of-bus/2020-09-01/18 2 September 2020,01:55:25 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:55:25 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:55:25 prefect.write_delta_lookup_table INFO file_list --> [] 2 September 2020,01:55:25 prefect.write_delta_lookup_table WARNING No transaction data in path - s3a://heb-stg-emd-store-raw-ora.pemd/firehose-emd-str-opstmt-srt-cd/2020-09-01/18 2 September 2020,01:55:25 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:55:25 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:55:25 prefect.write_delta_lookup_table INFO file_list --> [] 2 September 2020,01:55:25 prefect.write_delta_lookup_table WARNING No transaction data in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-frnt-end/2020-09-01/18 2 September 2020,01:55:25 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:55:25 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:55:26 prefect.write_delta_lookup_table INFO file_list --> ['20200901-180621308', '20200901-180621487', '20200901-180621645', '20200901-180621879'] 2 September 2020,01:55:38 prefect.write_delta_lookup_table INFO Delta write complete in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-retl-loc-stat-typ/delta_raw/ . Table name - emd.retl_loc_stat_typ 2 September 2020,01:55:59 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:55:59 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:55:59 prefect.write_delta_lookup_table INFO file_list --> [] 2 September 2020,01:55:59 prefect.write_delta_lookup_table WARNING No transaction data in path - s3a://heb-stg-emd-store-raw-ora.pemd/firehose-emd-location/2020-09-01/18 2 September 2020,01:55:59 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:55:59 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:55:59 prefect.write_delta_lookup_table INFO file_list --> [] 2 September 2020,01:55:59 prefect.write_delta_lookup_table WARNING No transaction data in path - s3a://heb-stg-emd-store-raw-ora.pemd/firehose-emd-retail-loc/2020-09-01/18 2 September 2020,01:55:59 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:55:59 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:56:00 prefect.write_delta_lookup_table INFO file_list --> ['20200901-180016582', '20200901-180017357', '20200901-180017804', '20200901-180018736', '20200901-180019364', '20200901-180020112', '20200901-180020786'] 2 September 2020,01:56:17 prefect.write_delta_lookup_table INFO Delta write complete in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-retl-loc-frmt/delta_raw/ . Table name - emd.retl_loc_frmt 2 September 2020,01:56:41 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:56:41 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:56:42 prefect.write_delta_lookup_table INFO file_list --> ['20200901-180813669', '20200901-180813908'] 2 September 2020,01:56:54 prefect.write_delta_lookup_table INFO Delta write complete in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-retl-loc-typ/delta_raw/ . Table name - emd.retl_loc_typ 2 September 2020,01:57:13 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:57:13 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:57:13 prefect.write_delta_lookup_table INFO file_list --> ['20200901-180419038', '20200901-180419499', '20200901-180420208', '20200901-180420547', '20200901-180420870', '20200901-180421119', '20200901-180421285', '20200901-180421591', '20200901-180421796', '20200901-180421971', '20200901-180422142', '20200901-180422347', '20200901-180423410', '20200901-180423647', '20200901-180423778', '20200901-180423915', '20200901-180424084', '20200901-180424228', '20200901-180424458', '20200901-180424666', '20200901-180424856', '20200901-180426043', '20200901-180426191', '20200901-180426376', '20200901-180426626', '20200901-180426750', '20200901-180426879', '20200901-180427037', '20200901-180427189', '20200901-180427385', '20200901-180427597', '20200901-180427719', '20200901-180427861', '20200901-180428060', '20200901-180428229', '20200901-180428469', '20200901-180428660', '20200901-180428837', '20200901-180429118', '20200901-180429344', '20200901-180429459', '20200901-180429602', '20200901-180429758', '20200901-180429912', '20200901-180430027'] 2 September 2020,01:57:49 prefect.write_delta_lookup_table INFO Delta write complete in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-dist/delta_raw/ . Table name - emd.dist 2 September 2020,01:58:59 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:58:59 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:58:59 prefect.write_delta_lookup_table INFO file_list --> ['20200901-182019711', '20200901-182020054', '20200901-182021115', '20200901-182021694', '20200901-182022141', '20200901-182022648', '20200901-182022879', '20200901-182023024', '20200901-182023237', '20200901-182023387', '20200901-182023528', '20200901-182023737', '20200901-182024008', '20200901-182024227', '20200901-182024420', '20200901-182024598', '20200901-182024834', '20200901-182024949', '20200901-182025081'] 2 September 2020,01:59:20 prefect.write_delta_lookup_table INFO Delta write complete in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-financial-div/delta_raw/ . Table name - emd.financial_div 2 September 2020,01:59:56 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,01:59:56 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': Starting task run... 2 September 2020,01:59:57 prefect.write_delta_lookup_table INFO file_list --> ['20200901-181015656', '20200901-181015991', '20200901-181016199', '20200901-181016621', '20200901-181016964', '20200901-181017160', '20200901-181017422', '20200901-181018264', '20200901-181018725', '20200901-181018912', '20200901-181019090', '20200901-181019276', '20200901-181019384', '20200901-181019575', '20200901-181019757', '20200901-181019969', '20200901-181020103', '20200901-181020268', '20200901-181020383', '20200901-181020757', '20200901-181020916', '20200901-181021029', '20200901-181021156', '20200901-181021315', '20200901-181021412', '20200901-181021567', '20200901-181021753', '20200901-181021898', '20200901-181022136', '20200901-181022254', '20200901-181022409', '20200901-181022711', '20200901-181022914', '20200901-181023129', '20200901-181023305', '20200901-181023414', '20200901-181023525', '20200901-181023660', '20200901-181023830', '20200901-181024006', '20200901-181024136', '20200901-181024378', '20200901-181024573', '20200901-181024677', '20200901-181024806', '20200901-181024903', '20200901-181025030', '20200901-181025138', '20200901-181025619'] 2 September 2020,02:00:53 prefect.write_delta_lookup_table INFO Delta write complete in path - s3a://heb-stg-emd-store-raw-ora.pemd/emd-mkt-rgn/delta_raw/ . Table name - emd.mkt_rgn 2 September 2020,02:02:33 prefect.CloudTaskRunner INFO Task 'write_delta_lookup_table': finished task run for task with final state: 'Success' 2 September 2020,02:02:33 prefect.CloudTaskRunner INFO Task 'final_task': Starting task run... 2 September 2020,02:02:34 prefect.final_task INFO Task successfully completed for all EMD tables for datetime - 2020-09-01/18 2 September 2020,02:02:34 prefect.CloudTaskRunner INFO Task 'final_task': finished task run for task with final state: 'Success' 2 September 2020,02:02:34 prefect.CloudFlowRunner INFO Flow run SUCCESS: all reference tasks succeeded