Data Engineering

Use when loading data into BigQuery from CSV, JSON, Avro, Parquet files, Cloud Storage, or local files. Covers bq load command, source formats, schema detection, incremental loading, and handling parsing errors.

For PMs and team leads who need to understand which key people their roadmap secretly depends on.Maps single points of failure in delivery, cross-functional dependencies, and knowledge concentration risks.Use when planning projects, assessing delivery risk, onboarding, or trying to understand why things keepgetting stuck on the same people.Keywords: dependencies, roadmap risk, bottleneck, single point of failure, key person, delivery risk,project planning, resource planning, who do we depend on, knowledge silos, bus factor

Deploy and manage cloud infrastructure, containerize applications with Docker, orchestrate with Kubernetes, implement CI/CD pipelines, and monitor systems. Use for cloud deployment, DevOps practices, infrastructure management, and scaling.

Эксперт Airbyte. Используй для настройки ETL/ELT пайплайнов, коннекторов, синхронизации данных и data pipelines.

ALWAYS use this skill when users ask to "create a skill", "make a skill for...", "add a new skill", or similar requests. This skill guides the creation of effective skills in the bigquery-etl repository that extend Claude's capabilities with specialized knowledge for BigQuery ETL workflows, Mozilla data platform conventions, or telemetry analysis. CRITICAL - First checks for conflicts with existing skills and recommends using/updating existing skills when appropriate. Do NOT attempt to create skills without invoking this skill first.

Analyze GitHub Actions CI failures for the current branch. Use when the user asks about CI status, test failures, build errors, GitHub Actions problems, or mentions checking CI, workflow failures, or pipeline status.

Automatically reference and validate schemas from both legacy data sources and medallion layer data sources (bronze, silver, gold) before generating PySpark transformation code. This skill should be used proactively whenever PySpark ETL code generation is requested, ensuring accurate column names, data types, business logic, and cross-layer mappings are incorporated into the code.

Use this skill when writing or updating SQL queries (query.sql) or Python ETL scripts (query.py) following Mozilla BigQuery ETL conventions. ALWAYS checks for and updates existing tests when modifying queries. Coordinates downstream updates to schemas and tests. Works with bigquery-etl-core, metadata-manager, and sql-test-generator skills.

Run comprehensive code quality checks including TypeScript typecheck, ESLint linting, Prettier formatting, and Markdown validation. Auto-fixes formatting issues in agent mode or provides read-only checks for CI pipelines. Use when: (1) Before committing code changes, (2) In CI/CD pipelines for automated quality gates, (3) After making significant code changes, (4) When preparing code for review, (5) When ensuring code meets quality standards, (6) For type checking, linting, formatting, and markdown validation, (7) In pre-commit hooks, or (8) For automated quality gates before merging. Triggers: finalize, code quality, typecheck, lint, format, check code, quality check, run checks, pre-commit, before commit, CI checks, validate code.

Create and manage long-running background processes with start/stop/status controls, logging, and monitoring. Use for batch processing jobs, data pipelines, continuous services, or any long-running tasks.

Processes images with Sharp, the high-performance Node.js library for resizing, converting, and optimizing images. Use when building image pipelines, generating thumbnails, or optimizing uploads server-side.

Designs and implements Extract, Transform, Load pipelines for data processing

Generate Apache Airflow ETL pipelines for government websites and document sources. Explores websites to find downloadable documents, verifies commercial use licenses, and creates complete Airflow DAG assets with daily scheduling. Use when user wants to create ETL pipelines, scrape government documents, or automate document collection workflows.

End-to-end testing for Blazor applications using Playwright for .NET. Use when writing E2E tests, creating test infrastructure, testing MudBlazor components, handling Blazor WebAssembly loading, debugging test failures, or setting up CI/CD test pipelines.

Master Docker containerization, Kubernetes orchestration, CI/CD pipelines, Infrastructure as Code (Terraform), and cloud platforms. Learn deployment automation, monitoring, scaling, and build reliable, scalable infrastructure for production systems.

Build new data ingestion providers following the FF Analytics registry pattern. This skill should be used when adding new data sources (APIs, files, databases) to the data pipeline. Guides through creating provider packages, registry mappings, loader functions, storage integration, primary key tests, and sampling tools following established patterns.

Search 76M+ patents using BigQuery (fast, zero setup) or PatentsView API (US patents with rich metadata).

Discover and introspect SQL Server database schemas dynamically. Use this skill when you need to understand table structures, column types, primary/foreign key relationships, or when adding new entities to the extraction pipeline.

Data import/export and validation pipeline for MyDetailArea. Handles bulk operations, CSV/Excel processing, data migration, validation rules, error handling, and audit trails. Supports vehicle inventory import, order batch updates, contact imports, and custom data transformations. Use when implementing bulk data operations, migration from legacy systems, or scheduled exports.

DevOps automation for Rust projects. CI/CD pipelines, container builds,deployment automation, and infrastructure as code. Optimized for GitHubActions and Cloudflare deployment.