We will use the following file format definition. This helps Snowflake understand the format of the CSV files Beat provides. It will be referenced in many

CREATE OR REPLACE FILE FORMAT BEAT_CSV_FORMAT
  TYPE = CSV
  COMPRESSION = ZSTD
  PARSE_HEADER = TRUE
  FIELD_DELIMITER = ','
  TRIM_SPACE = TRUE
  FIELD_OPTIONALLY_ENCLOSED_BY = '"'
  REPLACE_INVALID_CHARACTERS = TRUE
  DATE_FORMAT = 'AUTO'
  TIME_FORMAT = 'AUTO'
  TIMESTAMP_FORMAT = 'AUTO'
  NULL_IF = ('NULL','')
  BINARY_FORMAT = 'BASE64';

This file format crucially does a few things that are not standard in Snowflake:

• It treats the first line as a header line.
• It understands that empty strings should be treated as NULL values.
• It decompresses ZStandard-compressed files.
• Binary data is understood to be base-64 encoded.