Metrics

This skill provides comprehensive documentation for setting up Prometheus monitoring infrastructure. It covers installation via Helm and Docker, scrape configuration with static and dynamic service discovery, recording rules for pre-computed metrics, alert rules for availability and resource monitoring, and validation procedures using promtool.

This skill provides comprehensive guidance for creating production-ready Grafana dashboards. It includes JSON configuration templates for API monitoring, infrastructure, and database dashboards, along with best practices for panel types, variables, alerts, and dashboard provisioning using Terraform or Ansible.

This skill helps you implement comprehensive evaluation strategies for LLM applications. It covers automated metrics like BLEU, ROUGE, and BERTScore for measuring text quality, LLM-as-judge patterns for using stronger models to evaluate outputs, human evaluation frameworks with inter-rater agreement calculations, and statistical A/B testing for comparing model variants with proper significance testing.