VictoriaMetrics做Prometheus长期远程存储

VictoriaMetrics是什么请查看官网介绍：https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html

Prometheus是什么请查看官网介绍：https://prometheus.io/docs/introduction/overview/

规划如下

192.168.136.144 centos7	VictoriaMetrics , Prometheus , grafana
192.168.136.151 ubuntu20.4	node-export
192.168.136.153 ubuntu20.4	node-export

VictoriaMetrics = vm
Prometheus = plmxs

部署VictoriaMetrics

cd /usr/local/
mkdir vm
tar -xf victoria-metrics-amd64-v1.78.0.tar.gz -C /usr/local/vm
cd /usr/local/vm
mv victoria-metrics-prod victoria-metrics

#启动vm
nohup ./victoria-metrics -retentionPeriod=30d >& ./vm.log &
ss -luntp | grep "8428" 

部署Prometheus

tar -xf prometheus-2.35.0.linux-amd64.tar.gz -C /usr/local/
cd /usr/local/prometheus-v2.35

编辑配置文件,编辑前记得先备份。

vim prometheus.yml

global:
  scrape_interval:     15s
scrape_configs:
  - job_name: 'linux-system'
    static_configs:
    - targets: ['192.168.136.151:9100']
    - targets: ['192.168.136.153:9100']
    relabel_configs:   # 通过relabeling替换从__address__中提取IP信息
    - source_labels:  ['__address__']
      regex: '(.*):(.*)'
      action: replace
      target_label: 'ip'
      replacement: '${1}'
remote_write:    # 存储到远程VM存储
  - url: http://127.0.0.1:8428/api/v1/write   #vm的url
    queue_config:    # 如果Prometheus抓取指标很大，可以加调整queue，但是会提高内存占用
      max_samples_per_send: 10000
      capacity: 20000
      max_shards: 30

启动Prometheus

nohup ./prometheus --config.file="prometheus.yml" --storage.tsdb.retention=30d >& ./plmxs.log &
ss -luntp|grep "9090"
#浏览器访问http://192.168.136.144:9090

部署node-export

tar -xf node_exporter-1.3.1.linux-amd64.tar.gz -C /usr/local/
cd /usr/local/node_exporter-1.3.1.linux-amd64/
nohup ./node_exporter >& ./node-export.log &
ss -luntp|grep "9100"

部署grafana-9

wget https://dl.grafana.com/enterprise/release/grafana-enterprise-9.0.3-1.x86_64.rpm --no-check-certificate
yum -y install grafana-enterprise-9.0.3-1.x86_64.rpm
systemctl start grafana-server.service && systemctl status grafana-server.service

浏览器访问http://192.168.136.144:3000

拆分

如果不需要plmxs，直接用vm替换的话，只需要把plmxs的配置文件拷贝至vm启动目录，vm启动时指定plmxs的yaml文件即可

kill $plmxs-pid
#直接启动vm
nohup ./victoria-metrics -retentionPeriod=30d -promscrape.config=prometheus.yml >& ./vm.log &

log输出如下