Hi @Andreas.Motl,
yesterday I inserted a new control unit in Veglie but i can’t see it. Did I do something wrong?
My control unit can sometimes be seen on the map and others not.
But is the measurement interval important?
How many seconds do you recommend me to insert?
Thank you
a.
Dear Andrea,
after checking our infrastructure and running the production of the station list manually like
luftdatenpumpe stations --network=ldi --reverse-geocode --target=${rdbms_uri} --country=IT --progress
we receive errors like
2019-10-29 13:47:31,444 [luftdatenpumpe.geo ] ERROR : Reverse geocoding II failed: GeocoderUnavailable: Service not available. lat=44.68971596721, lon=10.63927710056
2019-10-29 13:47:32,444 [luftdatenpumpe.source.common ] ERROR : Failed computing humanized name for station Munch({'station_id': 16653, 'position': Munch({'latitude': 44.68971596721, 'longitude': 10.63927710056, 'altitude': 59.0, 'country': 'IT', 'exact_location': 1, 'indoor': 0, 'geohash': 'spzw4pe23ngg'})})
Traceback (most recent call last):
File "/opt/luftdatenpumpe/luftdatenpumpe/source/common.py", line 107, in enrich_station
country_code=station.position.get('country'),
File "/opt/luftdatenpumpe/luftdatenpumpe/geo.py", line 55, in resolve_location
location = reverse_geocode(latitude, longitude, country_code)
File "</opt/luftdatenpumpe/.venv3/lib/python3.7/site-packages/decorator-4.4.0-py3.7.egg/decorator.py:decorator-gen-1>", line 2, in reverse_geocode
File "/opt/luftdatenpumpe/.venv3/lib/python3.7/site-packages/dogpile.cache-0.7.1-py3.7.egg/dogpile/cache/region.py", line 1272, in get_or_create_for_user_func
should_cache_fn, (arg, kw))
File "/opt/luftdatenpumpe/.venv3/lib/python3.7/site-packages/dogpile.cache-0.7.1-py3.7.egg/dogpile/cache/region.py", line 879, in get_or_create
async_creator) as value:
File "/opt/luftdatenpumpe/.venv3/lib/python3.7/site-packages/dogpile.cache-0.7.1-py3.7.egg/dogpile/lock.py", line 186, in __enter__
return self._enter()
File "/opt/luftdatenpumpe/.venv3/lib/python3.7/site-packages/dogpile.cache-0.7.1-py3.7.egg/dogpile/lock.py", line 93, in _enter
generated = self._enter_create(value, createdtime)
File "/opt/luftdatenpumpe/.venv3/lib/python3.7/site-packages/dogpile.cache-0.7.1-py3.7.egg/dogpile/lock.py", line 179, in _enter_create
return self.creator()
File "/opt/luftdatenpumpe/.venv3/lib/python3.7/site-packages/dogpile.cache-0.7.1-py3.7.egg/dogpile/cache/region.py", line 839, in gen_value
created_value = creator(*creator_args[0], **creator_args[1])
File "/opt/luftdatenpumpe/luftdatenpumpe/geo.py", line 119, in reverse_geocode
raise ValueError('Reverse geocoding failed for lat={}, lon={}'.format(latitude, longitude))
ValueError: Reverse geocoding failed for lat=44.68971596721, lon=10.63927710056
So, we conclude something is fishy there. We will have to look into that problem.
With kind regards,
Andreas.
P.S.: For reverse geocoding the list of stations, we are using https://nominatim.openstreetmap.org/. It might either happen that some API semantics changed or that the program started violating the Nominatim Usage Policy in any way.
1 Like
Running luftdatenpumpe locally on my workstation successfully yields two stations in Veglie, Italy:
LDI station 19604
{
"station": {
"station_id": 19604,
"position": {
"latitude": 40.34,
"longitude": 17.964,
"altitude": 43.1,
"country": "IT",
"exact_location": 0,
"indoor": 0,
"geohash": "srhv2z8vx913"
},
"location": {
"place_id": 136685134,
"licence": "Data \u00a9 OpenStreetMap contributors, ODbL 1.0. https://osm.org/copyright",
"osm_type": "way",
"osm_id": 258406334,
"lat": "40.3398711314178",
"lon": "17.9640732926525",
"display_name": "Veglie, Lecce, Puglia, 73015, Italia",
"boundingbox": [
"40.3397004",
"40.3411381",
"17.9637731",
"17.9695617"
],
"address": {
"country_code": "IT",
"country": "Italia",
"state": "Puglia",
"county": "Lecce",
"postcode": "73015",
"town": "Veglie",
"city": "Veglie",
"road": "unknown"
}
},
"name": "Veglie, Lecce, Puglia, IT"
}
LDI station 19575
{
"station": {
"station_id": 19575,
"position": {
"latitude": 40.31988460214,
"longitude": 17.96343773603,
"altitude": 36.3,
"country": "IT",
"exact_location": 1,
"indoor": 0,
"geohash": "srhv2u0867mb"
},
"location": {
"place_id": 142613583,
"licence": "Data \u00a9 OpenStreetMap contributors, ODbL 1.0. https://osm.org/copyright",
"osm_type": "way",
"osm_id": 288569743,
"lat": "40.3198817298648",
"lon": "17.9634360846187",
"display_name": "Veglie, Lecce, Puglia, 73045, Italia",
"boundingbox": [
"40.3196403",
"40.3220242",
"17.9597578",
"17.963856"
],
"address": {
"country_code": "IT",
"country": "Italia",
"state": "Puglia",
"county": "Lecce",
"postcode": "73045",
"town": "Veglie",
"city": "Veglie",
"road": "unknown"
}
},
"name": "Veglie, Lecce, Puglia, IT"
}
Update: Running luftdatenpumpe on our servers works again without further ado so we have been able to investigate further.
Most probably, we’ve identified the problem. The records about station_id=19575 in the PostGIS database are missing an appropriate sensor_type identifier.
weatherbase=> select station_id, name, sensor_type from ldi_network where station_id in (19604, 19575);
station_id | name | sensor_type
------------+---------------------------+-------------
19575 | Veglie, Lecce, Puglia, IT |
19575 | Veglie, Lecce, Puglia, IT |
19604 | Veglie, LE, PUG, IT | BME280
19604 | Veglie, LE, PUG, IT | SDS011
(4 rows)
Now, we will have to investigate why this might have happened. Do you have any idea about this?
Running luftdatenpumpe stations --network=ldi --station=19575 --reverse-geocode does yield appropriate sensor type identifiers, see BME280 and SDS011.
[
{
"station_id": 19575,
"name": "Veglie, Lecce, Puglia, IT",
"position": {
"latitude": 40.31988460214,
"longitude": 17.96343773603,
"altitude": 36.3,
"country": "IT",
"exact_location": 1,
"indoor": 0,
"geohash": "srhv2u0867mb"
},
"location": {
"place_id": 142613583,
"licence": "Data \u00a9 OpenStreetMap contributors, ODbL 1.0. https://osm.org/copyright",
"osm_type": "way",
"osm_id": 288569743,
"lat": "40.3198817298648",
"lon": "17.9634360846187",
"display_name": "Veglie, Lecce, Puglia, 73045, Italia",
"boundingbox": [
"40.3196403",
"40.3220242",
"17.9597578",
"17.963856"
],
"address": {
"country_code": "IT",
"country": "Italia",
"state": "Puglia",
"county": "Lecce",
"postcode": "73045",
"town": "Veglie",
"city": "Veglie",
"road": "unknown"
}
},
"sensors": [
{
"sensor_id": 33217,
"sensor_type_name": "BME280",
"sensor_type_id": 17
},
{
"sensor_id": 33216,
"sensor_type_name": "SDS011",
"sensor_type_id": 14
}
]
}
]
Dear Andrea,
After upgrading luftdatenpumpe on our production server to its most recent version the other day, we missed to recreate the central database view ldi_network through luftdatenpumpe/luftdatenpumpe/target/rdbms.py at 0.18.2 · earthobservations/luftdatenpumpe · GitHub. In turn, the database field name sensor_type has been deprecated and replaced by sensor_type_id resp. sensor_type_name [1].
In order to account for this change, we will have to update our Grafana dashboards. On [2], you will be able to see both of your stations already. Enjoy!
With kind regards,
Andreas.
[1] luftdatenpumpe/CHANGES.rst at 0.18.2 · earthobservations/luftdatenpumpe · GitHub
1 Like
After adjusting things accordingly, station 19575 is now also on the map for Italy again.
https://weather.hiveeyes.org/grafana/d/ZKf6O2TWk/luftdaten-info-map-for-italy
1 Like