Databases

Lakes & Warehouses

1. What is a DWH? a comprehensive guide

2. DataLakeHouse - Firebolt comparison with Snowflake vs Databricks.

   1. Delta lake is a data lake that can store raw unstructured, semi-structured, and structured data. When combined with Delta Engine it becomes a data lakehouse.

3. What is SnowFlake, 2 - Snowflake decouples the storage and compute functions, which means organizations that have high storage demands but less need for CPU cycles, or vice versa, don’t have to pay for an integrated bundle that requires them to pay for both. Users can scale up or down as needed and pay for only the resources they use.

   1. get started with SF

4. data mart

   1. talend on data marts + 3 types (dependent, independent, hybrid)

   2. (good) netsuite on data marts - the three types ^ + structures (star, snowflake, denormalized) + comparisons

   3. basic intro

5. Data Lake

   1. monitoring health status at scale using great expectations and spark

Comparisons

1. Data Lake vs Data Warehouse

   

2. Top 5 differences between DL & DWH

3. Amazon on DL vs DWH

   

4. Snowflake vs Delta Lake vs Fire Bolt - "Databricks Delta Lake and Delta Engine is a lakehouse. You choose it as a data lake, and for data lakehouse-based workloads including ELT for data warehouses, data science and machine learning, even static reporting and dashboards if you don’t mind the performance difference and don’t have a data warehouse.

   Most companies still choose a data warehouse like Snowflake, BigQuery, Redshift or Firebolt for general-purpose analytics over a data lakehouse like Delta Lake and Delta Engine because they need performance.

   But it doesn’t matter. You need more than one engine. Don’t fight it. You will end up with multiple engines for very good reasons. It’s just a matter of when. "

5. Snowflake vs Amazon Redshift

6. Snowflake Intro and demo

7. The three pillars - snowflake

8. Snowflake vs redshift on medium

9. SF vs RS

10. RS vs BQ

11. SF vs BQ

Use Cases

1. Hunters on their architecture, airflow, snowflake, snowpipe, flink, rockdb, cluster optimization during ingestion, monitoring metrics, cost.

Snowflake

1. (good) Guides

2. getting started with SF tasks - sql or procedures, schedules, B-tree tasks.

3. cost

ClickHouse

1. open source database for real time apps and analytics - think snowflake open source

Feature engineering

1. Feature engineering in snowflake

Data lake Table Formats

Apache Iceberg

1. A short Intro

2. A Primer

3. Benchmarking Delta vs Iceberg vs Hudi

4. How we migrated our production data lake to iceberg

5. How we reduced our cost by 90%

6. Top 5 Features

Databricks Delta Lake

1. integrating delta lake into other platforms

Vector Databases

1. Gartner - Innovation Insight: Vector Databases

2. Chroma - AI native open source embedding database, github