Staff Backend Engineer - Second Horizon

About Grafana Labs and Role Overview

Grafana Labs is the company behind Grafana Cloud, a fully managed observability platform trusted by over 10,000 organizations including Anthropic, Bloomberg, NVIDIA, Microsoft, and Salesforce. The company champions an open-source heritage combined with a global remote work culture spanning more than 40 countries. This role is fully remote within the U.S. or Canada (excluding Quebec residents).

The Staff Backend Engineer position is part of a pioneering skunkworks project aimed at embedding observability across the enterprise. The focus is on creating an AI-native data intelligence system that supplies AI agents with reliable, managed access to enterprise data context such as metadata, lineage, and quality indicators. This role involves end-to-end backend system development encompassing ingestion, indexing, API design, and integration with AI agents.

Key Responsibilities

• Design and build core backend services for context ingestion, indexing, retrieval orchestration, API endpoints, configuration management, and administrative operations.
• Develop a scalable multi-tenant SaaS architecture incorporating tenant isolation, usage monitoring, quotas, audit logs, background processing, and defined service boundaries.
• Implement APIs and interfaces enabling AI agents, command-line tools, and internal users to access relevant context information complete with provenance and confidence metrics.
• Collaborate across product and infrastructure teams to balance rapid prototyping with long-term system reliability.
• Own operation of built services using observability practices including metrics, logging, tracing, alerting, and dashboards to ensure system health.
• Contribute to overall technical direction involving architecture, service design, storage strategies, API contracts, deployment, and engineering standards.
• Communicate effectively within highly dynamic, collaborative environments and contribute to cross-team initiatives.
• Take end-to-end ownership of AI-driven software solutions to ensure scalability, maintainability, and alignment with real user workflows.

Required Skills and Experience

• Robust engineering experience delivering production software and systems with minimal supervision.
• Practical familiarity with AI technologies including large language models (LLMs), prompt engineering, and GenAI-powered applications.
• Comfort with iterative development: releasing prototypes, gathering feedback, and refining solutions.
• Proven initiative to define scope amid ambiguity and drive projects toward impactful outcomes.
• Experience integrating observability tools to monitor and troubleshoot distributed systems.
• Background in cloud-native environments such as AWS, GCP, or Azure.
• Strong collaborative mindset and effective communication skills.

Preferred Qualifications

• Experience with agent-based frameworks or coordinating multi-agent workflows.
• Familiarity with data analysis platforms (Looker, Tableau, PowerBI, Snowflake, DataBricks) or data engineering tooling.

Compensation and Benefits

The base salary range for Canadian candidates is CAD 186,368 to CAD 223,642 annually, subject to level, skills, and experience evaluated in the hiring process. Additional compensation components include equity and potential bonuses.

Benefits include fully remote work, global cultural diversity, transparent communication, innovation encouragement, open source involvement, empowered teams, career progression, approachable leadership, and comprehensive leave policies including 30 days annually with additional shutdown days.

Equal Opportunity and Inclusivity

Grafana Labs promotes an inclusive workplace appreciating diversity across all protected categories. The recruitment process may use AI-assisted tools alongside manual review to align candidate qualifications with role requirements.