Making Puppet host reports in Foreman available in Grafana
Where I work, we use Puppet to manage our growing infrastructure of Linux machines and Loki together with Grafana to analyse log files.
When the Puppet agent runs on a host it creates something called a „Puppet config report“ which includes all steps taken by the Puppet agent to ensure the required configuration as specified in the Puppet catalog for that host. In our case, these reports are then sent back to Foreman as we’re using that on the Puppet server as a frontend.
To analyse these files better, we wanted to push these reports to Loki. Thing is, they are stored in the Foreman database in several tables (as good database design goes).
Here’s a small script to turn those reports into files which in turn can be scraped by Promtail or the likes to get them to Loki and analyse them using Grafana assuming you’re using PostgreSQL as your Foreman database:
#!/usr/bin/env bash
# Get report ids from today
REPORT_IDS=$(echo "select r.id from reports r where date_trunc('day', r.reported_at) = date_trunc('day', now())" | su - postgres -c "psql -A -t foreman")
for REPORT_ID in ${REPORT_IDS}
do
REPORT_FILENAME=$(echo "select h.name || '-' || to_char(r.reported_at,'YYYY-MM-DD\"T\"HH24:MI:SS') || '.log' from reports r join hosts h on r.host_id=h.id where r.id=${REPORT_ID}" | su - postgres -c "psql -A -t foreman")
cat <<EOT | su - postgres -c "psql -A -t foreman" > "${REPORT_FILENAME}"
WITH levels (id,level) as (values (2, 'notice'))
SELECT r.created_at || ' ' || '[' || levels.level || '] ' || h.name || ' ' || s.value || ' ' || m.value
FROM messages m
JOIN logs l on m.id=l.message_id
JOIN sources s on s.id=l.source_id
JOIN levels on levels.id=l.level_id
JOIN reports r on l.report_id = r.id
JOIN hosts h on h.id = r.host_id
WHERE report_id=${REPORT_ID}
ORDER by l.id
EOT
done
Diese Website verwendet Akismet, um Spam zu reduzieren. Erfahre mehr darüber, wie deine Kommentardaten verarbeitet werden.
Calendar
M | D | M | D | F | S | S |
---|---|---|---|---|---|---|
1 | 2 | 3 | 4 | 5 | 6 | |
7 | 8 | 9 | 10 | 11 | 12 | 13 |
14 | 15 | 16 | 17 | 18 | 19 | 20 |
21 | 22 | 23 | 24 | 25 | 26 | 27 |
28 | 29 | 30 | 31 |
Archive
- Januar 2024
- Dezember 2023
- April 2021
- März 2021
- September 2020
- Dezember 2019
- November 2019
- Oktober 2019
- Juli 2019
- Juni 2019
- Mai 2019
- April 2019
- März 2019
- September 2018
- August 2018
- Juli 2018
- März 2018
- Januar 2018
- Dezember 2017
- September 2017
- März 2017
- Februar 2017
- Januar 2017
- August 2016
- Mai 2016
- Dezember 2015
- November 2015
- August 2015
- März 2015
- Dezember 2014
- September 2014
- August 2014
- Juli 2014
- Februar 2014
- Oktober 2013
- September 2013
- August 2013
- Juli 2013
- Juni 2013
- Mai 2013
- April 2013
- November 2012
- Oktober 2012
- September 2012
- August 2012
- Juni 2012
- Mai 2012
- März 2012
- Februar 2012
- Januar 2012
- November 2011
- Juli 2011
- Juni 2011
- März 2011
- Februar 2011
- Januar 2011
- Dezember 2010
- November 2010
- April 2010
- Februar 2010
Schreibe einen Kommentar