Skip to content
Back to portfolio

AWS and Snowflake modernization

Healthcare Data Modernization Platform

A governed payer-data platform supporting claims, eligibility, member, provider, pharmacy, finance, and operational reporting.

50M+

healthcare records processed

35%

latency reduction

30%

compute cost savings

The failure mode

What had to be solved

Legacy healthcare workloads needed a reusable cloud platform with controlled ingestion, certified analytics layers, strong security, and recoverable production operations.

01

Healthcare Sources

02

AWS Standardization

03

Orchestration

04

Snowflake Marts

Architecture

Source Landing

Database extracts, REST APIs, partner files, and CDC feeds land in encrypted S3 zones with audit columns and schema capture.

AWS Standardization

AWS Glue and PySpark standardize high-volume claims, eligibility, member, provider, and pharmacy datasets.

Airflow Orchestration

Apache Airflow orchestrates ingestion, validation, transformation, publication, retries, SLAs, and backfills.

Snowflake Serving

Snowflake and dbt provide dimensional marts with conformed dimensions, SCD Type 2 history, snapshots, tests, and documentation.

Key Engineering Decisions

01

Recoverable ingestion for sensitive payer data

Partner files, database extracts, APIs, and CDC feeds all had different failure modes and audit requirements.

Built reusable ingestion patterns with schema capture, audit columns, encryption, retries, quarantine handling, and recoverable incremental loads.

02

Certification before analytics publication

Claims and payment discrepancies could not flow silently into reporting layers.

Added row-count, payment-total, duplicate, referential-integrity, and freshness reconciliation before datasets were certified for analytics.

03

Dimensional marts instead of report-specific logic

Healthcare teams needed consistent definitions across claims, member, provider, pharmacy, finance, and operations reporting.

Modeled Snowflake/dbt fact and dimension layers with conformed dimensions, SCD Type 2 history, snapshots, tests, and documentation.

Operating Spec

Scale

  • 50M+ healthcare records
  • Multi-terabyte tables
  • Claims, eligibility, member, provider, pharmacy, finance, and operations domains

Quality

  • Row count reconciliation
  • Claim/payment total checks
  • Referential integrity
  • Duplicate detection
  • Freshness checks

Reliability

  • Airflow retries and SLAs
  • Quarantine handling
  • Recoverable incremental loads
  • Root-cause remediation for data discrepancies

Tools

AWS S3AWS GluePySparkApache AirflowSnowflakedbtTerraformCloudWatchIAM/KMS