Instrument a service for observability

Help me instrument a service for observability.
Inputs: service; stack; existing tooling.

1. Instrument critical user requests across metrics and traces.
2. control cardinality, sampling and sensitive data.
3. connect alerts to actionable diagnosis and recovery.

Deliver: Questions to answer, Plan, Traces, Metrics, Logs, Code changes, Dashboards and alerts.
Clarify critical gaps, separate facts from assumptions, and do not invent data, citations or completed actions.

My material: [fill in here]

Before you start

Provide service; stack; existing tooling.