What bronze, silver, and gold mean
Bronze is immutable-ish raw land: files and tables as they arrived, with ingestion timestamps and source metadata. Silver applies business rules—types, deduplication, joins across sources—so analysts trust column meanings. Gold shapes data for specific outcomes: feature stores for ML, forecast-ready time series, executive metrics.
Teams adopt medallion layers to stop training models on the same messy export CRM admins pulled manually. When ML engineers know gold v2 added regional splits, everyone retrains against the same definition.
How Heimdall Lake implements each layer
Bronze ingestion supports CSV uploads, relational database extracts, and unstructured zip archives with schema inference. Files land in a catalog you can browse, preview, and tag—no external object-store console required.
Silver curation happens in the Lake UI and Heimdall Lab: filter bronze tables, join entities, compute rolling metrics, and save curated views back to silver. Lab notebooks use pandas and plotly when visual exploration beats form-based transforms.
Gold publishing promotes silver snapshots optimized for downstream products. Heimdall ML and Forecast read gold directly—no export to Parquet on a shared drive, no brittle pipeline to a feature store.
- Bronze: raw ingest with lineage and previews
- Silver: curated, conformed datasets and notebook transforms
- Gold: modeling-ready tables for ML, Forecast, and Loop
Managed medallion vs DIY cloud storage
Rolling your own medallion stack on S3 or ADLS means IAM policies, Glue or Hive catalogs, Spark or dbt jobs, and monitoring for failed partitions. That is the right investment at huge scale with platform engineers on staff.
Heimdall Lake targets teams that need medallion discipline at GB-to-TB scale with UI-first operations. Ingest Monday, silver by Wednesday, gold live for Friday's model review—a cadence SMB and mid-market companies actually hit.
Governance, collaboration, and ML readiness
Because Lake shares an account with Lab, ML, Forecast, Loop, and Resolve, permissions and dataset names stay consistent across products. Data owners curate silver; model owners consume gold; nobody maintains a spreadsheet of 'which CSV is canonical.'
When auditors ask what trained model 14, you point to gold dataset version 3 derived from silver view 7 sourced from bronze ingest 22—a chain visible inside the platform.
Getting started with medallion on Heimdall
Register at heimdallapp.org, create a workspace, and upload your first bronze files or connect a database. Document silver transformations—even lightweight ones like dropping nulls and standardizing country codes—before promoting gold.
Explore warehouse use cases such as unstructured archive search or database consolidation in our use case hub. Pair Lake with Lab when you need Python, and with ML when gold is ready for predictions.