Access to InfluxDB or Grafana instance with data from luftdaten.info for Italy

Air quality data
, ,
1

Dear Andreas,

did you find the time to discuss the opportunity giving us a playground environment, or an account on one of your Grafana instances?

If you need some time to discuss this theme internally, could you give us a timing?

Thank’s
Martino

2

Dear Martino,

thanks for asking.

We will ramp up a respective playground environment for you, as we already planned do to so in order to satisfy other requests coming to us from different directions.

We kindly ask you to give us some days to get things sorted in this matter.

With kind regards,
Andreas.

3

Dear Martino,

We have been pretty busy maintaining our baseline infrastructure [1,2,3], supported our community members with other data ingests [4] and will have to look into some details of our software components around Luftdatenpumpe [5,6] again.

So, we would like to apologize that we haven’t been able to find any time to fulfill your requests during the last week, please bear with us.

With kind regards,
Andreas.

[1] Maintenance of our bare-metal servers
[2] Services offline while upgrading operating system - Maschinenraum / Machine room - Hiveeyes
[3] Serverstörung wegen defektem CPU-Lüfter - Maschinenraum / Machine room - Hiveeyes
[4] Problem backfilling a large historical dataset into InfluxDB
[5] Luftdatenpumpe
[6] Missing data of my luftdaten.info sensor with LDI station id 19604 from Veglie, Italy on the Grafana map

4

Dear Andreas,
don’t worry. We can wait :wink:
The project with Arpa will start in January 2020, in this two month we will have some meeting with them and I think we can talk about you. If you have a short presentation to give them, I think could be useful.
Thank you
martino

1 Like
5

Dear Andreas,
waiting for an instance on your grafana we are working to a local server with influxdb and grafana instance.
we are experienced these problems:

  • some city are not displayed with a city filter
  • the address taken from open street maps contains several errors. how do you solve this problem?
  • your dashboard here: Grafana is not working, you know why?
    here the link to our server, if you want take a look, and give us some advice.Grafana
    thank you very much
    ps the code for the dasboard here Grafana will be available?
1 Like
6

Hi there,

Status quo

All right. Sorry for not being able to provide you a tenant on our system in time!

That looks nice!

Problems

Where and how do you apply this filter? To luftdatenpumpe or within Grafana? We would need some examples to be able to investigate further [1].

Do we actually solve it? Using the reverse-geocoded information from OpenStreetMap’s Nominatim service is currently the only thing luftdatenpumpe can do for us. So, the outcome should be the same for all of us.

However, I would be interested to know more details about these problems you are observing. While I can’t promise how and when we would be able to mitigate them, it would nevertheless be useful to have some pointers at hand for doing so some time in the future.

This looks like a leftover issue from a schema migration process we had to perform recently. Can I humbly ask you take care of that, @wtf?

Outlook

I am humbly diverting this to @wtf [1:1].

Good luck with everything!

Cheers,
Andreas.

  1. I will be traveling until 7th of December or so. Sorry! ↩︎ ↩︎

7

I think directly on grafana. Actually i don’t have direct access to the platform, it is made by another guy.
I can ask to @wtf also the code for the dashboard https://weather.hiveeyes.org/grafana/d/ioUrPwQiz/luftdaten-info-verlauf?

thank you

8

Let’s see … error-message says: "pq: column "ldi_sensors.sensor_type" must appear in the GROUP BY clause or be used in an aggregate function".

Something with sensor_type … smells like an issue when the queries for polluting our dashboards-variables are parsed. Let’s see in variables definitions (under Dashboard Settings > Variables): ah, $ldi_station_sensortype is using sensor_type:

SELECT sensor_type_name AS __value, concat( sensor_type_name, ' |', CAST(COUNT(sensor_type_name) AS text), '|' ) AS __text FROM ldi_sensors GROUP BY sensor_type_name ORDER BY sensor_type

Ah, there is no GROUP BY sensor_type … so the ORDER BY can’t work therefore. Let’s add sensor_type to GROUP BY:

SELECT sensor_type_name AS __value, concat( sensor_type_name, ' |', CAST(COUNT(sensor_type_name) AS text), '|' ) AS __text FROM ldi_sensors GROUP BY sensor_type_name, sensor_type ORDER BY sensor_type

Seems fixed to me, can you confirm?

Ugh, ah … we haven’t released this yet. Cause this particular dashboard is done for a customer I will have a small diplomatic chitchat with them on this first, at end of next week. But I may assume: besides the politness this would be just a matter of time or publicity, we both agreed on making this open source, but project is still in development.

1 Like
9

Have you already seen the dashboards “shipped with luftdatenpumpe”? The corresponding one should be “Trend”:

1 Like
10

Yeah. “Trend” is the new “Verlauf” [1] and it is shipped with luftdatenpumpe already. Some template variables have to be interpolated appropriately through the deployment process of luftdatenpumpe itself as outlined within Deploy LDI Grafana artefacts with Lufdatenpumpe.

Have fun!

  1. The deployment of weather.hiveeyes.org is really ancient, we are still running Grafana 5 there ;[. ↩︎

11

Thanks for fixing this!

Feel free to invite him to our place here. He is welcome.

12

Yes now is working, thank you.

ok i will have a look on Trend default dashboard.

I invited him but he’s very busy at the moment. I ask deeper the issue, and, generally speaking, he want a unique indication for every geographical site. For example italian region of “Emilia Romagna” is indicated as:
EMR, Emilia Romagna, Pianura Padana.
At the moment he’s thinking on rewriting on another DB the right address.
what do you think about this?

1 Like
13

3 posts were merged into an existing topic: Discussing ambiguity with locations in Italy

14

Same everywhere ;].

Nominatim already yields two identifiers for a location called place_id and osm_id. If you won’t find them back in the PostgreSQL database already, we might want to add them. Please let us know.

We might think about investigating the issue in order to aim at fixing it within Luftdatenpumpe. The problems you are observing will probably also hit others and in that manner, we can do better for the whole community out there.

In order to get the ball rolling, we created a topic to discuss the situation and a corresponding issue on GitHub. Feel free to comment on both of them as you like.

15

We got in common that it’s okay when I send you a current working copy, but beware: wo don’t consider the current state as reliable/stable/peer-reviewed/checked/more-than-a-proof-on-concept/here-be-dragons/draft/prototype/name-it! What Grafana & InfluxDB-versions are you using? We’ve just discovered some things to adapt …

1 Like
16

No problem with the work in progress status! We love challenges! For the Grafana and InfluxDB versions here:

  • Grafana 6.5.1
  • InfluxDB 1.7.9

We also have a question on how to Count and display values exceeding thresholds from air quality measurements within Grafana.

TX

17

I just sent you the current working-state of the dashboard privately. Have fun with it, probably ask more general technical/engineering question here (more public) and please understand that my priority for this work belongs to VMM, but it will emit good stuff for your adaptions for sure, too :slight_smile:

1 Like