Secret Network
WebsiteDiscordGithub
  • 👋INTRODUCTION
    • Secret Network Introduction
    • Secret Network Techstack
      • Private transactions - A quick summary
      • Blockchain Technology
        • Cosmos Basics
        • Tendermint
        • Cosmos SDK
        • IBC
        • CosmWasm
      • Privacy Technology
        • Encryption - Key Management
          • Overview
          • Key Derivation & Encryption Techniques
          • The Initialization Of Secret Network
          • Full Node Boostrap
          • Contract State Encryption
          • Transaction Encryption
          • Consensus seed rotation
        • Trusted Execution Environments  (TEE) —  Intel SGX
          • How Secret Network Uses SGX
          • SGX-SPS Security & Reliabillity
          • Remote Attestation
          • Trusted & Untrusted Core
          • Sealing
        • Private smart contracts - overview
  • 💻Development
    • 🏠Getting Started
      • Setting Up Your Environment
        • Cargo.toml Best Practices (Crates vs Dependencies)
      • Compile and Deploy
      • Running the Application
      • Compile and Deploy on Secret testnet (best for Javascript devs)
      • Fullstack dApp Integration
    • 🤓Secret Contracts
      • Secret Contracts & CosmWasm
        • Framework overview
        • Secret Contract Components
          • Instantiation Message
          • Execution Message
          • Query Message
          • Deps/DepsMut
          • Storage
            • Prefixed Storage
            • Singleton
            • Keymap
            • Append Store
            • Best practices
        • CosmWasm vs Secret CosmWasm
      • Secret Tokens (SNIP-20)
      • Contract - module call
      • Secret contract - Design Space/Features
        • Secret Contracts introduction
        • Gas/Fee usage
        • TPS and scalability
        • Privacy Essentials
        • Access Control
          • Viewing Keys
          • Permits
        • Trusted and untrusted data
        • Secret-VRF - on-chain Randomness
        • Privacy design
          • Mitigate privacy risks - full guide
          • Gas Evaporation & Tracking
        • Confidential Computing Layer
        • Fiat/Crypto Onboarding
        • Account abstraction
        • Fee abstraction
        • Wallet support
        • Bridge (messaging/tokens)
        • IBC (Hooks, PFM, Wasm)
        • Price Oracles
        • Auto Restaking
      • Permissioned Viewing
        • Viewing Keys
        • Permits
      • Cross Contract Communication
      • Submessages
        • get_contract_code_hash
      • Randomness API - Secret VRF
        • Native On-chain randomness
        • Randomness over IBC
      • Execution Finalization
      • Factory Contracts
      • Contract Migration
        • Manual - < v1.11
        • Native - from v1.11
      • Cross-deploy Vanilla CW and Secret Contracts
      • Testing Secret Contracts
        • Unit Tests
        • Continuous Integration
        • Datatype Handling - Uint, floats etc.
    • 👀Secret Contract - Reference, Guides, Examples
      • Starter guide - Millionaire's Problem
      • Reference Contracts
      • Open source dApps
      • Tools & Libraries
        • Network interaction SDKs
          • Secret.js (JavaScript)
          • SecretPy (Python)
          • SecretK (Kotlin)
          • Secret.NET
            • Snippets
              • Deploying a Contract
              • Send Native Coin
              • Query a Contract
              • Create a new Wallet
              • Permits
              • SNIP20
              • SNIP721
          • Shade.Js
        • LocalSecret - Devnet docker
        • Smart contract - Tools/Utils
          • Secret Toolkit
          • CW-Plus
          • Fadroma - SC framework
          • Hidden Gems
          • Other
            • Secret IDE
            • Polar
    • 🖼️Frontend Development
      • Getting Started with SecretJS
        • Usage Examples
          • Sending Queries
          • Sending Messages
          • Contract Migration
          • Wallet Integrations
          • Query Permits
          • SNIP20 (SCRT Tokens)
          • SNIP721 (Secret NFTs)
      • Feegrant
        • Understanding Feegrant allowances
        • Grant allowances
        • Using grant allowances to execute transactions
        • Using the Fee Grant Faucet
    • 📬Deployment Addresses/ API Endpoints
      • Secret (SNIP 20) token contracts list
        • SNIP Asset naming guidelines
      • Connecting to the Network
        • API Endpoints Mainnet (Secret-4)
        • API Endpoints Testnet (Pulsar-3)
        • Usage examples
        • Comparison of endpoint types
      • Create your own SNIP-25 IBC Token
  • Secret AI
    • 🤝Introduction
    • 🏗️Architecture
    • 👩‍💻Secret AI SDK
      • Setting Up Your Environment
      • Running the Application
    • 🪙Economics
    • Smart Contract Reference
      • SubscriptionManager
      • WorkerManager
      • RewardsManager
  • 🌐SecretVM - Confidential Virtual Machines
    • 🤝Introduction
    • 🏗️Architecture
    • ☑️Attestation
      • What is Attestation
      • Obtaining Attestation Data
      • Attestation Report - Key Fields
      • Chain of Trust
    • 🏁Launching a SecretVM
    • 🛠️Managing SecretVM Lifecycle
    • ✅Verifying a SecretVM
    • 💡Best Practices for Developers
    • 📖Glossary
  • 🔓Confidential Computing Layer
    • 🧑‍🚀IBC Developer Toolkit
      • Basics
        • Overview
        • Cross-chain Messaging with IBC Hooks
          • Functions, Methods, and Data Structures
          • Typescript SDK
          • IBC-Hooks
        • IBC Relaying with Go Relayer
      • Usecases
        • Storing Encrypted Data on Secret Network
          • Key-Value store Developer Tutorial
        • Secret VRF for IBC with IBC-Hooks
        • Confidential Voting
        • Sealed Bid Auctions
      • Supported Networks
        • Mainnet
        • Testnet
    • 🤝Ethereum (EVM) Developer Toolkit
      • Basics
        • Overview
        • Connecting Metamask to Secret Network
        • SecretPath + Reown integration
        • Cross-chain Messaging
          • SecretPath
            • Architecture Overview
            • Detailed Architecture
            • SecretPath Developer Tutorials
            • Public EVM Gateway Architecture
            • How to deploy SecretPath on your chain
          • Axelar GMP
            • Architecture Overview
            • Axelar GMP Developer Tutorial
      • Usecases
        • Storing Encrypted Data on Secret Network
          • Key-Value store Developer Tutorial
        • Sealed Bid Auction
          • Sealed Bid Auction Developer Tutorial
        • Confidential Voting
          • Confidential Voting Developer Tutorial with SecretPath
        • VRF
          • Implementing VRF into any EVM Contract
          • VRF Developer Tutorial
          • Performance figures of SecretVRF vs competitors
          • Using encrypted payloads for VRF
          • Converting from Chainlink VRF to Secret VRF in four steps
        • Confidential Document Sharing
        • Tokens
          • From EVM to Secret
      • Supported Networks
        • EVM
          • EVM Mainnet
          • EVM Testnet
          • Gateway Contract ABI
        • Secret Gateway
          • SecretPath mainnet (secret-4) contracts
          • SecretPath testnet (pulsar-3) contracts
    • 🙌Solana Developer Toolkit
      • Usecases
        • Storing Encrypted Data on Secret Network
          • Key-value Store Developer Tutorial
        • VRF
          • VRF Developer Tutorial
      • Program IDs
        • Solana Mainnet & Testnet
        • Gateway Contract IDL
  • 🤫Overview, Ecosystem and Technology
    • 🚀Secret Network Overview
      • The SCRT coin
      • Private Tokens
      • Use Cases
        • Decentralized Finance (DeFi)
        • Art And Digital Media
        • Gaming
        • Data Privacy
        • Payments And Transactions
        • Communication
      • The technology
      • History
      • Roadmap (Core development)
        • Secret 2.0
      • Where To Buy SCRT?
      • Using the Testnet
    • 🐸Ecosystem Overview
      • Wallets
      • Applications
      • Explorers & tools
      • Funding
        • SCRT Labs Grants
        • Dilutive funding/VC raise
        • Community Pool
        • Developer bounties
          • SCRT Labs bounties [on-hold]
          • CCBL [on-hold]
          • CCR [On-hold]
        • Application specific
          • Shade Grants
      • Contributors & Entities
        • Validators
        • SCRT Labs
        • Secret Foundation
        • Secret Committees
          • Support
          • Governance
      • Secret Network Dictionary
  • 🔧Infrastructure
    • 🔓Use SecretCLI
      • Secretcli vs. Secretd
      • Install
      • Configuration
      • Address Types
      • Key Types
      • Generating Keys
      • Viewing Keys
      • Query Transactions
      • Send Tokens
      • Multisig Keys
      • Multisig Transactions
      • Transaction Broadcasting
      • Fees & Gas
      • Fee Distribution
      • Secret Contracts
      • Slashing
      • Minting
      • Delegating
      • Restake
      • Nodes
      • Governance
        • Creating Governance Proposals
        • Query Proposals
        • Deposits
        • Voting
    • 🔐Use Ledger hardware wallet
      • 🔐Ledger with SecretCLI
    • 🖥️Running a node/validator
      • Setting up a node/validator
        • Hardware setup
          • Hardware Compliance
          • VPS/Bare-Metal Compliance
            • Leaseweb Setup
            • PhoenixNAP Setup
            • Psychz Setup
            • nForce Setup
            • Vultr Setup
            • OVHCloud Setup
            • Microsoft Azure Setup
          • Patching your Node
          • Enclave verification
          • Registration troubleshooting
        • Testnet Setup
          • Install SGX
          • Install secretd
          • Setup Full Node
          • Testnet State Sync
          • Becoming a Testnet Validator
          • Installing CLI & Creating A New Address
        • Mainnet Setup
          • Install SGX
          • Install secretd
          • Setup Full Node
          • Quicksync / Snapshot
          • Statesync
          • Becoming A Validator
          • Installing CLI & Creating A New Address
      • Maintaining a node/validator
        • Slashing information
        • Migrating a Validator
        • Troubleshooting
        • Validator Backup
        • Server security
          • SSH authentication
          • Server configuration
          • Uncomplicated-Firewall (UFW)
          • Local CLI
        • Node Monitoring
          • Prometheus
            • Environment Preperation
            • Install Node Exporter
            • Install Prometheus
            • Configuring Prometheus
          • Grafana
            • Install Grafana
            • Grafana Dashboard
            • Next Steps
          • Docker
            • Install Docker
            • Configuration
            • Start Containers
            • Grafana Dashboard
            • Application Ports
            • Stop Containers
          • Goaccess
            • Install Goaccess
            • Setup Goaccess
        • Helpful commands
          • Query Validators
          • Bond Tokens
          • Withdraw Rewards
          • Query Delegations
          • Unbond Tokens
          • Query Unbonding-Delegations
          • Redelegate Tokens
          • Query Redelegations
          • Query Parameters
          • Query Pool
          • Query Delegations To Validator
      • API Noderunning
        • Running Multiple Nodes on the Same Server
        • Node Loadbalancing using Nginx
          • Setup Nginx
          • Example Nginx config
        • Using Auto heal to improve cluster uptime for Nginx
      • Sentry and Archive nodes
        • Mantlemint
        • Sentry Nodes
        • Archive Nodes
    • ⛓️IBC Relayers
      • Hermes
      • RLY
      • IBC channel database
    • 🆙Upgrade Instructions
      • v1.13
      • v1.12
      • v1.11
      • v1.10
      • v1.9
      • v1.8
      • v1.7
      • Shockwave Omega v1.6
      • v1.5
      • Shockwave Delta v1.4
      • Shockwave Delta v1.4 (Testnet)
      • Shockwave Alpha v1.3
      • Cosmovisor
      • Vulcan Network Upgrade (OLD)
    • ☠️Postmortems
      • SNIP-20 leaks
      • xApic
      • Secpk-Verifications Bloat
      • Earn Contract Exploit
      • Testnet Halt 95
    • ✍️Contribute to the documentation
      • Report Bugs
      • Suggest Enhancements
      • First Contribution Guide
      • Pull Request Templates
        • Update Documentation
        • Bug Fix
        • Improve Performance
        • Change Functionality
      • Style Guide
    • 🌊Versioning & Changelog
      • Secret Network v1.13
      • Secret Network v1.12
      • Secret Network v1.11
      • Secret Network v1.10
      • Secret network v1.9
      • Secret Network v1.7/1.8
      • Secret Network v1.6
      • Secret Network v1.5
      • Secret Network v1.4 (CosmWasm 1.0)
Powered by GitBook
On this page

Was this helpful?

Edit on GitHub
Export as PDF
  1. Infrastructure
  2. Running a node/validator
  3. Maintaining a node/validator
  4. Node Monitoring
  5. Prometheus

Configuring Prometheus

PreviousInstall PrometheusNextGrafana

Last updated 2 years ago

Was this helpful?

Prior to using Prometheus, it needs basic configuring. Thus, we need to create a configuration file named prometheus.yml

The configuration file of Prometheus is written in which strictly forbids to use tabs. If your file is incorrectly formatted, Prometheus will not start. Be careful when you edit it.

Open the file prometheus.yml in a text editor:

sudo nano /etc/prometheus/prometheus.yml

Prometheus’ configuration file is divided into three parts: global, rule_files, and scrape_configs.

In the global part we can find the general configuration of Prometheus: scrape_interval defines how often Prometheus scrapes targets, evaluation_interval controls how often the software will evaluate rules. Rules are used to create new time series and for the generation of alerts.

The rule_files block contains information of the location of any rules we want the Prometheus server to load.

The last block of the configuration file is named scape_configs and contains the information which resources Prometheus monitors.

Our file should look like this example:

global:  
  scrape_interval:     15s  
  evaluation_interval: 15s
  
rule_files:
  # - "first.rules"  
  # - "second.rules"
  
scrape_configs:  
  - job_name: 'prometheus'    
    scrape_interval: 5s    
	static_configs:      
	  - targets: ['localhost:9090']

The global scrape_interval is set to 15 seconds which is enough for most use cases.

We do not have any rule_files yet, so the lines are commented out and start with a #.

In the scrape_configs part we have defined our first exporter. It is Prometheus that monitors itself. As we want to have more precise information about the state of our Prometheus server we reduced the scrape_interval to 5 seconds for this job. The parameters static_configsand targets determine where the exporters are running. In our case it is the same server, so we use localhost and the port 9090.

As Prometheus scrapes only exporters that are defined in the scrape_configs part of the configuration file, we have to add Node Exporter to the file, as we did for Prometheus itself.

We add the following part below the configuration for scraping Prometheus:

- job_name: 'node_exporter'  
  scrape_interval: 5s  
  static_configs:    
    - targets: ['localhost:9100']

Overwrite the global scrape interval again and set it to 5 seconds. As we are scarping the data from the same server as Prometheus is running on, we can use localhost with the default port of Node Exporter: 9100.

If you want to scrape data from a remote host, you have to replace localhost with the IP address of the remote server.

Set the ownership of the file to our Prometheus user:

sudo chown prometheus:prometheus /etc/prometheus/prometheus.yml

Our Prometheus server is ready to run for the first time.

Running Prometheus

  1. Start Prometheus directly from the command line with the following command, which executes the binary file as our Prometheus user:

sudo -u prometheus /usr/local/bin/prometheus --config.file /etc/prometheus/prometheus.yml --storage.tsdb.path /var/lib/prometheus/ --web.console.templates=/etc/prometheus/consoles --web.console.libraries=/etc/prometheus/console_libraries

The server starts displaying multiple status messages and the information that the server has started:

level=info ts=2018-04-12T11:56:53.084000977Z caller=main.go:220 msg="Starting Prometheus" version="(version=2.2.1, branch=HEAD, revision=bc6058c81272a8d938c05e75607371284236aadc)"
level=info ts=2018-04-12T11:56:53.084463975Z caller=main.go:221 build_context="(go=go1.10, user=root@149e5b3f0829, date=20180314-14:15:45)"
level=info ts=2018-04-12T11:56:53.084632256Z caller=main.go:222 host_details="(Linux 4.4.127-mainline-rev1 #1 SMP Sun Apr 8 10:38:32 UTC 2018 x86_64 scw-041406 (none))"
level=info ts=2018-04-12T11:56:53.084797692Z caller=main.go:223 fd_limits="(soft=1024, hard=65536)"
level=info ts=2018-04-12T11:56:53.09190775Z caller=web.go:382 component=web msg="Start listening for connections" address=0.0.0.0:9090
level=info ts=2018-04-12T11:56:53.091908126Z caller=main.go:504 msg="Starting TSDB ..."
level=info ts=2018-04-12T11:56:53.102833743Z caller=main.go:514 msg="TSDB started"
level=info ts=2018-04-12T11:56:53.103343144Z caller=main.go:588 msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml
level=info ts=2018-04-12T11:56:53.104047346Z caller=main.go:491 msg="Server is ready to receive web requests."

Open your browser and type http://IP.OF.YOUR.SERVER:9090 to access the Prometheus interface. If everything is working, we end the task by pressing on CTRL + C on our keyboard.

If you get an error message when you start the server, double-check your configuration file for possible YAML syntax errors. The error message will tell you what to check.

The server is working now, but it cannot yet be launched automatically at boot. To achieve this, we have to create a new systemd configuration file that will tell your OS which services should it launch automatically during the boot process.

sudo nano /etc/systemd/system/prometheus.service

The service file tells systemd to run Prometheus as prometheus and specifies the path of the configuration files.

Copy the following information in the file and save it, then exit the editor:

[Unit]
Description=Prometheus Monitoring
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus \
--config.file /etc/prometheus/prometheus.yml \
--storage.tsdb.path /var/lib/prometheus/ \
--web.console.templates=/etc/prometheus/consoles \
--web.console.libraries=/etc/prometheus/console_libraries
ExecReload=/bin/kill -HUP $MAINPID

[Install]
WantedBy=multi-user.target

To use the new service, reload systemd:

sudo systemctl daemon-reload

We enable the service so that it will be loaded automatically during boot:

sudo systemctl enable prometheus

Start Prometheus:

sudo systemctl start prometheus

Your Prometheus server is ready to be used.

We have now installed Prometheus to monitor your instance. Prometheus provides a basic web server running on http://your.server.ip:9000 that provide access to the data collected by the software.

Tip: For all information about the configuration of Prometheus, you may check the .

🔧
🖥️
YAML
configuration documentation