File Transfer — Illustrated

This guide outlines how file transfers between Cardlytics and Publishers are implemented using Amazon S3 Replication, supporting both AWS KMS (Key Management Service) and SSE-S3 (Amazon S3-managed keys) encryption.

How It Works

File transfers are performed via Amazon S3 Replication:

• Publishers send files to CDLX: via replication from Publisher’s S3 bucket to CDLX’s Source bucket.
• CDLX sends files to Publishers: via replication from CDLX’s Destination bucket to Publisher’s Source bucket.

Separate setups are repeated for Pre-Prod and Prod environments.

For reference: AWS S3 Replication Documentation

Encryption Options

• KMS Encryption (SSE-KMS): Used when customer-managed encryption keys are required.
• SSE-S3 Encryption: Used for simplified setup without customer-managed keys.

Setup Overview

Cardlytics Will Provide:

• AWS Account Number
• Incoming S3 Bucket ARN (CDLX_Source_BUCKET_NAME)
• Outgoing S3 Bucket ARN (CDLX_Destination_BUCKET_NAME)
• CDLX IAM Principal ARN (CDLX_IAM_ARN)
• CDLX KMS Key ARN (CDLX_KMS_ARN) — KMS encryption only

Publisher Will Provide:

• Publisher Source S3 Bucket ARN
• Publisher Destination S3 Bucket ARN
• IAM Role ARN (IAM_ROLE_ARN) for replication
• Publisher KMS Key ARN (KMS_ARN) — KMS encryption only

Bucket Configuration

Destination_BUCKET (Publisher → CDLX)

• Versioning: Enabled
• Encryption: SSE-KMS (provide KMS_ARN) or SSE-S3
• Bucket Policy: Enforce HTTPS only

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowSSLRequestsOnly",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::Destination_BUCKET_NAME/*",
        "arn:aws:s3:::Destination_BUCKET_NAME"
      ],
      "Condition": {
        "Bool": {
          "aws:SecureTransport": "false"
        }
      }
    }
  ]
}

Source_BUCKET (CDLX → Publisher)

• Copy configuration from Destination_BUCKET
• Additional Bucket Policy for replication:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "SetPermissionsForObjects",
      "Effect": "Allow",
      "Principal": {"AWS": "CDLX_IAM_ARN"},
      "Action": ["s3:ReplicateObject", "s3:ObjectOwnerOverrideToBucketOwner"],
      "Resource": "arn:aws:s3:::Source_BUCKET_NAME/*"
    },
    {
      "Sid": "SetPermissionsOnBucket",
      "Effect": "Allow",
      "Principal": {"AWS": "CDLX_IAM_ARN"},
      "Action": ["s3:GetBucketVersioning", "s3:PutBucketVersioning"],
      "Resource": "arn:aws:s3:::Source_BUCKET_NAME"
    }
  ]
}

IAM Role for Replication

Trust Policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {"Service": "s3.amazonaws.com"},
      "Action": "sts:AssumeRole"
    }
  ]
}

IAM Policy

• For KMS encryption:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowKMSKeyUsage",
      "Effect": "Allow",
      "Action": ["kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:DescribeKey"],
      "Resource": ["CDLX_KMS_ARN"]
    },
    {
      "Sid": "DestinationBucketPermissions",
      "Effect": "Allow",
      "Action": ["s3:ListBucket", "s3:GetObjectVersionTagging", "s3:GetObjectVersionAcl", "s3:GetObjectVersionForReplication", "s3:GetReplicationConfiguration", "s3:GetObjectRetention", "s3:GetObjectLegalHold", "s3:ReplicateObject", "s3:ReplicateTags", "s3:ObjectOwnerOverrideToBucketOwner"],
      "Resource": ["arn:aws:s3:::Destination_BUCKET_NAME/*", "arn:aws:s3:::Destination_BUCKET_NAME"]
    },
    {
      "Sid": "ExternalBucketPermissions",
      "Effect": "Allow",
      "Action": ["s3:GetObjectVersionTagging", "s3:ReplicateObject", "s3:ReplicateTags", "s3:ObjectOwnerOverrideToBucketOwner"],
      "Resource": ["arn:aws:s3:::CDLX_Source_BUCKET_NAME/*"],
      "Condition": {
        "StringLikeIfExists": {
          "s3:x-amz-server-side-encryption": ["aws:kms"],
          "s3:x-amz-server-side-encryption-aws-kms-key-id": ["CDLX_KMS_ARN"]
        }
      }
    }
  ]
}

• For SSE-S3 encryption: Same as above but remove all KMS-related statements and conditions.

Replication Rule (Destination Bucket)

Configure replication from Publisher’s Destination_BUCKET_NAME to CDLX_Source_BUCKET_NAME:

• Destination: CDLX_Source_BUCKET_NAME
• Account ID: CDLX_ACCOUNT_NUMBER
• Change object ownership: Enabled
• IAM Role: IAM_ROLE_ARN
• Encryption:
  • For KMS: select "Replicate objects encrypted with AWS KMS" and provide CDLX_KMS_ARN
  • For SSE-S3: no action required

File Path Structure

Use the following format for file naming:
{Root}/{YYYY-MM-DD}/CDLX_<file-type>_<date>_<epoch>_1of2.jsonl

Summary

• KMS Encryption offers fine-grained control and secure key management.
• SSE-S3 simplifies the setup by removing the need to manage KMS keys.
• Both approaches use S3 Replication with enforced SecureTransport and object ownership transfer to ensure secure, automated file delivery between Publisher and Cardlytics.