Grafana architect

Backend code structure

Grafanas backend is written in GO using sqlite3/mysql or postgres as storage for dashboards, users etc. When Grafana was born there didnt exist much guides or direction for how to write medium sized application. So there are parts of Grafana code base that didn't quite pan out as we wanted. More about that under current rewrites! :)

Central folders of Grafanas backend

folder	description
/pkg/api	Http Handlers and routing. Almost all handler funcs are global which is something we would like to improve in the future. Handlers should be assosicated with a struct that refers to all depedencies. Ex bus.
/pkg/cmd	The binaries that we build. Grafana-server and grafana-cli.
/pkg/components	Mixed content of packages that we copied into Grafana and packages we implemented ourself. The Purpose of this folders should be packages that are too big for util and doesnt belong somewhere else.
/pkg/infra	Packages in infra should be packages that are used in multiple places of Grafana without knowing anything about Grafana domain. E.g. Logs, Metrics, Traces.
/pkg/services	Packages in services are responsible for peristing domain objects and manage the relationship between domain objects. Services should communicate with each other using DI when possible. Most part of Grafana's codebase still relay on Global state for this. Any new features going forwards should use DI.
/pkg/tsdb	All backend implementations of the data sources in Grafana. Used by both Grafanas frontend and alerting.
/pkg/util	Small helper functions that are used in multiple parts of the codebase. Many functions are placed directly in the util folders which is something we want to avoid. Its better to give the util function a more descriptive package name. Ex errutil.

Central components of Grafanas backend

package	description
/pkg/bus	The bus! Described more below.
/pkg/models	This is where we keep our domain model. This package should not depend on any package outside standard library. (It does contain some refs within Grafana but that is something we should avoid going forward).
/pkg/registry	service management package.
/pkg/services/alerting	Grafanas alerting services. The alerting engine run in a seperate go routine and should not depend on anything else within Grafana.
/pkg/services/sqlstore	Currently where are database calls resides.
/pkg/setting	settings package for Grafana. Anything related to grafana global configuration should be dealt with in this package.

Testing

We value clean & readable code that is loosely coupled and covered by unit tests. This makes it easier to collaborate and maintain the code. In the sqlstore package we do database operations in tests and while some might say that's not suited for unit tests. We think they are fast enough and provide a lot of value.

The majority of our tests uses go convey but thats something we want to avoid going forward. For new tests we want to use standard library and testify/assert.

Import aliases

Currently there are import aliases for /pkg/models package but thats something we want to avoid going forward. Newly introduced code should refer explicitly to the models instead of using the alias m. Whenever changing existing code it's desired to remove the import aliases as well.

The Bus

The bus is our way to introduce indirection between the HTTP handlers and sqlstore (responsible for fetching data from the database). Http handlers and sqlstore don't depend on each other. They only depend on the bus and the domain model(pkg/models). This makes it easier to test the code and avoids coupling. More about this under current rewrite/refactorings

Services/Repositories

Services within Grafana should be self-contained and only talk to other parts of Grafana using the bus or repositories that have been made available through Grafana service registry. All services should register themselves to the registry package in an init function. Only registration should be done in the init function. Init functions should be avoided as much as possible.

When Grafana starts all init functions within the services will be called and register themselves. Grafana will then create a Graph of all dependencies and inject the services that other services depend on. This is solved with inject library in https://github.com/grafana/grafana/blob/master/pkg/cmd/grafana-server/server.go#L75

Provisionable*

All new features that require state should be possible to configure using config files. Ex data sources, alert notifiers, dashboards, teams etc. Today its only possible to provision data sources and dashboards but this is something we want to support all over Grafana.

 

 

 

Frontend: Code structure, architecture and plans for the future

 

Marcus Efraimsson edited this page on 14 Aug 2018 · 8 revisions

Frontend

The Grafana frontend is a single page application written in angular and react and sass. Built using webpack. Angular is the main framework that handles the URL -> component routing.

Structure

The frontend code lives in the public folder.

Location	Description
app/app.ts	This is where the angular app is bootstrapped.
app/sass	Frontend css styles
app/core	common components and services
app/features	components and services organized by feature
app/routes	URL -> angular or react component mapping
app/stores	Mobx stores
app/plugins	All built-in plugins live here (panels, data sources)
app/containers	React containers and container-specific components

Central components & services

• backendSrv: used for all backend API requests and data source requests.
• templateSrv: used by dashboard components to interpolate a string containing template variables
• timeSrv: used by dashboard components to interact with the dashboard time range
• dashboardSrv: used to get current dashboard and save / load actions.
• bridgeSrv: bridges React and angular world (especially around interacting with URL state)

Unit testing

We value clean & readable code that is loosely coupled and covered by unit tests. We use Jest for all of our JavaScript tests.

see our guidelines for more details.

React + Angular

We are in the process of migrating the complete code base to React. This will be a multi-year effort. The reasons for this work is to make sure Grafana remains a modern frontend application with clean and maintainable code. AngularJS has lots of issues that make complex HTML rendering expensive and hard to maintain. We also see React being a much simpler and easy framework to learn and work with. It will likely remain relevant for much longer than AngularJS and most importantly it will be far easier to attract talented frontend developers if we use this framework than stick with AngularJS forever.

Currently, some whole pages are written in React and some components are written in react. It is easy to use a react component in an angular component. The other way around is a bit trickier but possible.

In progress refactoring/rewrites

• Angular -> React rewrites

Future refactoring/rewrites

• Simplify code structure (move components out of app/core to app/components)
• Fix inconsistent naming of files. Angular and other js components use snake case file naming while react components use PascalCase.
• Make it possible to write panel and data source plugins in React.
• backendSrv and use of angular $http, need to migrate to fetch

Challenges for the future

• In browser integration testing
• Plugin regression testing (capture images of different states of panels maybe?)
• Dashboard url state handled via viewStateSrv is very messy.