Data Engineering

A comprehensive reference guide for ClickHouse database development. Provides SQL patterns for table design (MergeTree engines), query optimization techniques, data insertion patterns, materialized views, and common analytics queries including time series analysis, funnel analysis, and cohort analysis.

A comprehensive reference guide for optimizing Apache Spark jobs. Covers partitioning strategies, join optimization (broadcast, sort-merge, bucket joins), caching patterns, memory configuration, shuffle reduction techniques, and data format optimization with practical PySpark code examples.

This skill provides comprehensive documentation and best practices for building production MLOps pipelines. It covers the full ML lifecycle including data preparation, model training, validation, and deployment strategies with guidance on using orchestration tools like Airflow, Dagster, and Kubeflow.

A comprehensive data engineering skill that teaches production-ready Apache Airflow DAG patterns including TaskFlow API, dynamic DAG generation, sensors, branching logic, and error handling. Also covers dbt transformation patterns, Spark optimization techniques, and data quality frameworks with Great Expectations.