The Grafana DataFrame type

[Andreas.Motl]

Introduction

The people of Grafana Labs are reorganizing their code and introduce the DataFrame type to streamline the data shuffling under the hood, e.g. between table format and timeseries format.

DataFrame, our primary data model, has now a columnar layout. This will support easier frontend processing. The DataSource query interface has been updated to better support streaming. The result can now either return a Promise<result> or Observable<result>. Be on the lookout for more on live data streaming in the future!

– What’s new in Grafana v6.4: Share query results between panels
– Grafana 6.4.0 released - #2 by Andreas - Maschinenraum / Machine room - Hiveeyes

Background

The concept of using column-oriented data frames to shuffle data around has been popularized by R’s data.frame() and Pandas’ pandas.DataFrame already.

• data.frame function - RDocumentation
• pandas.DataFrame — pandas 2.1.4 documentation

Rationale

• Support other types besides series data (Histogram, Number Data, Generic Tables)
• Create a clearer API for future backend datasource plugins
• Simpler integration between frontend and backend

– Make DataFrame structure for datasource responses that is shared by backend and frontend · Issue #18085 · grafana/grafana · GitHub

Resources

• DataFrame filenames & organization · Issue #18822 · grafana/grafana · GitHub
• grafana/data: Reorganise code by dprokop · Pull Request #19136 · grafana/grafana · GitHub

Implementation

• grafana/packages/grafana-data/src/dataframe at main · grafana/grafana · GitHub