S3 Publisher File Transfer with SSE-S3

Solution Outline

This solution uses Amazon S3 Replication with Amazon S3-managed keys (SSE-S3) for encryption to ensure secure file transfer. It simplifies the setup by removing KMS key management while maintaining data security both at rest and in transit.

Setup Overview

Prod

Cardlytics Setup

Cardlytics will provide the Publisher with details for the destination S3 bucket that will receive files:

AWS Account Number:
Destination S3 Bucket ARN:
S3 Encryption: Enable SSE-S3 for automatic encryption of data at rest in both incoming and outgoing buckets.

Publisher Setup

The Publisher will configure the source S3 bucket, which will automatically replicate files to the CDLX destination bucket.

Source AWS Account ID:
Source S3 Bucket ARN:
Source IAM Role ARN Used for Replication:
Encryption: Default S3 encryption (SSE-S3).

The Publisher will request the above destination bucket details from Cardlytics.

Pre-Prod

Cardlytics Setup

Cardlytics will provide the Publisher with:

AWS Account Number:
Incoming S3 Bucket ARN:
S3 Encryption: Enable SSE-S3 for automatic encryption of data at rest in both incoming and outgoing buckets.

Publisher Setup

Source AWS Account ID:
Source S3 Bucket ARN:
Source IAM Role ARN Used for Replication:
Encryption: Default S3 encryption (SSE-S3).

The Publisher will request the same information from Cardlytics.

S3 File Path Setup

{Root}/{YYYY-MM-DD}/CDLX_<file-type>_<date>_<epoch>_1of2.jsonl

Setup Instructions

S3 Bucket Configurations

EGRESS_BUCKET (Publisher → CDLX)

Enable Bucket Versioning.
Enforce Secure Transport (HTTPS-only):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowSSLRequestsOnly",
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::EGRESS_BUCKET_NAME/*",
                "arn:aws:s3:::EGRESS_BUCKET_NAME"
            ],
            "Condition": {
                "Bool": {
                    "aws:SecureTransport": "false"
                }
            }
        }
    ]
}

INGRESS_BUCKET (CDLX → Publisher)

Replicate the same configuration and secure transport policy from the outgoing bucket.

IAM Role for Replication

Trust Policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "s3.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

IAM Policy (Simplified, No KMS Dependencies)

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "EgressBucketPermissions",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObjectVersionTagging",
                "s3:GetObjectVersionAcl",
                "s3:GetObjectVersionForReplication",
                "s3:GetReplicationConfiguration",
                "s3:GetObjectRetention",
                "s3:GetObjectLegalHold",
                "s3:ReplicateObject",
                "s3:ReplicateTags",
                "s3:ObjectOwnerOverrideToBucketOwner"
            ],
            "Resource": [
                "arn:aws:s3:::EGRESS_BUCKET_NAME/*",
                "arn:aws:s3:::EGRESS_BUCKET_NAME"
            ]
        },
        {
            "Sid": "ExternalBucketPermissions",
            "Effect": "Allow",
            "Action": [
                "s3:GetObjectVersionTagging",
                "s3:ReplicateObject",
                "s3:ReplicateTags",
                "s3:ObjectOwnerOverrideToBucketOwner"
            ],
            "Resource": [
                "arn:aws:s3:::CDLX_INGRESS_BUCKET_NAME/*"
            ]
        }
    ]
}

Replication Rule for Egress Bucket

Configure replication from the EGRESS_BUCKET_NAME to the CDLX_INGRESS_BUCKET_NAME.

Destination Account ID: CDLX_ACCOUNT_NUMBER
Destination Bucket: CDLX_INGRESS_BUCKET_NAME
Change Object Ownership: Enabled
IAM Role: IAM_ROLE_ARN

Summary

This configuration ensures:

Secure file transfers using HTTPS enforcement (SecureTransport).
Data encryption with SSE-S3, managed entirely by Amazon S3.
Simplified replication setup without KMS complexities.
Standard S3 replication and versioning for reliable, automated delivery.