There are regular upgrades to the Secret Network that require users and validators to update their systems. This section is dedicated to hosting upgrade instructions for Secret Network upgrades. There are instructions for:

• Shockwave Alpha

• Shockwave Delta

• Shockwave Delta (Testnet)

• v1.5

• Shockwave Omega

• v1.7

• v1.8

• v1.9

• v1.10

• v1.11

• v1.12

• Cosmovisor

• Vulcan Network Upgrade (OLD)

Secret Network v1.10 Instructions

⚠️ IMPORTANT NOTES ⚠️

• All coordination efforts will be done in the "SN Validators" Telegram group.

• Make sure to backup your validator before making any changes.

• Please read carefully before you begin the upgrade.

Upgrading Manually

Note: The below instructions assume default installation. DO NOT COPY-PASTE if config.toml is not in ~/.secretd/config/config.toml or if you modified /etc/systemd/system/secret-node.service. If you have modified /etc/systemd/system/secret-node.service, you will need to re-apply those changes post installation and pre service restart.

When the network reaches the halt height 10,186,400, you'll see this message in your node's log (journalctl -fu secret-node):

2:00PM ERR UPGRADE "v1.10" NEEDED at height: 10186400:
2:00PM ERR CONSENSUS FAILURE!!! err="UPGRADE \"v1.10\" NEEDED at height: 10186400

Then, the upgrade steps for v1.10 are:

# Backup your node's SGX secrets
cp -a /opt/secret/.sgx_secrets ~/sgx_secrets_backup
mkdir 19backup
cp ~/.secretd/.node/* 19backup/

# Figure out currently used db backend
DB_BACKEND=$(awk -F \" '/^db_backend =/{print $2}' ~/.secretd/config/config.toml)

# Get the v1.10 binaries
wget "https://github.com/scrtlabs/SecretNetwork/releases/download/v1.10.0/secretnetwork_1.10.0_mainnet_${DB_BACKEND}_amd64.deb"

# Verify the v1.10 binaries
echo '4c1818a6cc8b72e20e492741cb9fbed6567e8971333718f0787e8f182fda18a4 secretnetwork_1.10.0_mainnet_goleveldb_amd64.deb
f33f1494e6498974e13b205e94dd4c538707b8bc8dafbbc7153b79f491b31e22 secretnetwork_1.10.0_mainnet_rocksdb_amd64.deb' |
    grep "$DB_BACKEND" |
    sha256sum --check

# Stop the v1.9 node
sudo systemctl stop secret-node

# Install the v1.10 binaries
sudo apt install -y "./secretnetwork_1.10.0_mainnet_${DB_BACKEND}_amd64.deb"

# re-apply any systemd unit file customizations

# Restart the node
sudo systemctl restart secret-node

After restarting the node with v1.10, you should see INF applying upgrade "v1.10" at height: 10186400 in the logs (journalctl -fu secret-node). Once 67% of voting power comes online, you'll see blocks executing again.

Upgrading Automatically Using Cosmovisor

Cosmovisor is a new process manager for cosmos blockchains. It can make low-downtime upgrades smoother, as validators don't have to manually upgrade binaries during the upgrade, and instead can pre-install new binaries, and Cosmovisor will automatically update them based on on-chain SoftwareUpgrade proposals.

⚠️ Cosmovisor is still new and best practices for using it are not yet established. If you don't feel adventurous at this time, we recommend upgrading the manual way.

For instructions on how to setup Cosmovisor, go here.

Details of Upgrade Time

When the network reaches the halt height 10,186,400, the Secret Network blockchain will be halted and validators will need to take action to upgrade the chain to the secretd v1.10 binary (be it manually or automatically).

The proposal targets the upgrade proposal block to be 10,186,400, anticipated to be on Monday August 14, 2023 at 5:00PM UTC.

The upgrade is anticipated to take approx 30 minutes, during which time, there will not be any on-chain activity on the network.

In Case of an Upgrade Failure

In the event of an issue at upgrade time, we should coordinate via the "SN Validators" Telegram group.

If as a result of a software bug the network fails to produce new blocks with the v1.10 binaries, the SCRT Labs team will distribute a v1.9 binary with an empty v1.10 upgrade handler, which will allow the chain to revert to v1.9 while continuing to produce new blocks.

Secret Network v1.8 Emergency Upgrade Instructions

⚠️ IMPORTANT NOTES ⚠️

• All coordination efforts will be done in the "SN Validators" Telegram group.

• Make sure to backup your validator before making any chnages.

• Please read carefully before you begin the upgrade.

Upgrading Manually

Note: The below instructions assume default installation. DO NOT COPY-PASTE if config.toml is not in ~/.secretd/config/config.toml or if you modified /etc/systemd/system/secret-node.service. If you have modified /etc/systemd/system/secret-node.service, you will need to re-apply those changes post installation and pre service restart.

When the network reaches the halt height 7,760,000, you'll see this message in your node's log (journalctl -fu secret-node):

2:00PM ERR UPGRADE "v1.8" NEEDED at height: 7760000:
2:00PM ERR CONSENSUS FAILURE!!! err="UPGRADE \"v1.8\" NEEDED at height: 7760000

Then, the upgrade steps for v1.8 are:

# Backup your node's SGX secrets
cp -a /opt/secret/.sgx_secrets ~/sgx_secrets_backup
mkdir 17backup
cp ~/.secretd/.node/* 17backup/

# Figure out currently used db backend
DB_BACKEND=$(awk -F \" '/^db_backend =/{print $2}' ~/.secretd/config/config.toml)

# Get the v1.8 binaries
wget "https://github.com/scrtlabs/SecretNetwork/releases/download/v1.8.0/secretnetwork_1.8.0_mainnet_${DB_BACKEND}_amd64.deb"

# Verify the v1.8 binaries
echo 'd1bb747afd5cdbbd5330f37a8c1eeba1312cc891af19a1f589eb3e3f44630e23 secretnetwork_1.8.0_mainnet_goleveldb_amd64.deb
e377c67dd1760760fd3ada90517e55e446664d6372224f6558fe74decc1acb0d secretnetwork_1.8.0_mainnet_rocksdb_amd64.deb' |
    grep "$DB_BACKEND" |
    sha256sum --check

# Stop the v1.7 node
sudo systemctl stop secret-node

# Install the v1.8 binaries
sudo apt install -y "./secretnetwork_1.8.0_mainnet_${DB_BACKEND}_amd64.deb"

# re-apply any systemd unit file customizations

# Restart the node
sudo systemctl restart secret-node

After restarting the node with v1.8, you should see INF applying upgrade "v1.8" at height: 7760000 in the logs (journalctl -fu secret-node). Once 67% of voting power comes online, you'll see blocks executing again.

Upgrading Automatically Using Cosmovisor

Cosmovisor is a new process manager for cosmos blockchains. It can make low-downtime upgrades smoother, as validators don't have to manually upgrade binaries during the upgrade, and instead can pre-install new binaries, and Cosmovisor will automatically update them based on on-chain SoftwareUpgrade proposals.

⚠️ Cosmovisor is still new and best practices for using it are not yet established. If you don't feel adventurous at this time, we recommend upgrading the manual way.

For instructions on how to setup Cosmovisor, go here.

Details of Upgrade Time

When the network reaches the halt height 7,760,000, the Secret Network blockchain will be halted and validators will need to take action to upgrade the chain to the secretd v1.8 binary (be it manually or automatically).

The proposal targets the upgrade proposal block to be 7,760,000, anticipated to be on Saturday March 4, 2023 at 3:00PM UTC.

The upgrade is anticipated to take approx 30 minutes, during which time, there will not be any on-chain activity on the network.

In Case of an Upgrade Failure

In the event of an issue at upgrade time, we should coordinate via the "SN Validators" Telegram group.

If as a result of a software bug the network fails to produce new blocks with the v1.8 binaries, the SCRT Labs team will distribute a v1.7 binary with an empty v1.8 upgrade handler, which will allow the chain to revert to v1.7 while continuing to produce new blocks.

Secret Network Shockwave Omega Upgrade Instructions

⚠️ IMPORTANT NOTES ⚠️

• All coordination efforts will be done in the "SN Validators" Telegram group.

• Make sure to backup your validator before making any chnages.

• Please read carefully before you begin the upgrade.

Check your hardware compatiblity before the upgrade

To check whether your hardware will be able to register on the network, you can run the check-hw tool of the new release (or the latest RC version, if not available). You can find it in the Releases page.

Download the check-hw tar, extract and run:

tar -xzf chec_hw_<version>.tar.gz
cd check-hw
LOG_LEVEL=WARN ./check-hw

You should see some logs and if your hardware is compatible, you should receive at the end:

Platform Okay!

Upgrading Manually

When the network reaches the halt height 6,537,300, you'll see this message in your node's log (journalctl -fu secret-node):

1:25PM ERR UPGRADE "v1.6" NEEDED at height: 6537300:
1:25PM ERR CONSENSUS FAILURE!!! err="UPGRADE \"v1.6\" NEEDED at height: 6537300

Then, the upgrade steps for v1.6 are:

⚠️ Note: if you have modified your systemd unit file (/etc/systemd/system/secret-node.service) you will need to re-apply those changes post installation and pre service restart.

# Stop the v1.5 node, to make sure that your process manager isn't trying to restart it while you upgrade
sudo systemctl stop secret-node

# Get & verify secretd v1.6

# Check what database type you're currently using:
awk -F \" '/^db_backend =/{print $2}' ~/.secretd/config/config.toml

# Uncomment the right binary based on the database type: `rocksdb` or `goleveldb`

## goleveldb
# wget "https://github.com/scrtlabs/SecretNetwork/releases/download/v1.6.0/secretnetwork_1.6.0_mainnet_goleveldb_amd64.deb"
# echo "ce9ba85d346fa460ed3fc98871f2a254b269fafa835fc555c9184f6405d8c80a secretnetwork_1.6.0_mainnet_goleveldb_amd64.deb" | sha256sum --check

## rocksdb
# wget "https://github.com/scrtlabs/SecretNetwork/releases/download/v1.6.0/secretnetwork_1.6.0_mainnet_rocksdb_amd64.deb"
# echo "65f795069fc3f703ef3bc1e856cb7499f5474e7b7889ca7d83e085be8b5488f1 secretnetwork_1.6.0_mainnet_rocksdb_amd64.deb" | sha256sum --check

# Install v1.6 binaries
sudo apt install -y ./secretnetwork_1.6.0_mainnet_*_amd64.deb

# re-apply any systemd unit file customizations

# Restart the node
sudo systemctl restart secret-node

After restarting the node with v1.6, you should see INF applying upgrade "v1.6" at height: 6537300 in the logs (journalctl -fu secret-node). Once 67% of voting power comes online, you'll see blocks executing again.

Upgrading Automatically Using Cosmovisor

Cosmovisor is a new process manager for cosmos blockchains. It can make low-downtime upgrades smoother, as validators don't have to manually upgrade binaries during the upgrade, and instead can pre-install new binaries, and Cosmovisor will automatically update them based on on-chain SoftwareUpgrade proposals.

⚠️ Cosmovisor is still new and best practices for using it are not yet established. If you don't feel adventurous at this time, we recommend upgrading the manual way.

For instructions on how to setup Cosmovisor, go here.

Details of Upgrade Time

When the network reaches the halt height 6,537,300, the Secret Network blockchain will be halted and validators will need to take action to upgrade the chain to the secretd v1.6 binary (be it manually or automatically).

The proposal targets the upgrade proposal block to be 6,537,300, anticipated to be on Tuesday December 13, 2022 at 12:00PM UTC.

The upgrade is anticipated to take approx 30 minutes, during which time, there will not be any on-chain activity on the network.

In Case of an Upgrade Failure

In the event of an issue at upgrade time, we should coordinate via the "SN Validators" Telegram group.

If as a result of a software bug the network fails to produce new blocks with the v1.6 binaries, the SCRT Labs team will distribute a v1.5 binary with an empty v1.6 upgrade handler, which will essentially allow the chain to revert to v1.5 while continuing to produce new blocks.

Secret Network v1.13 Instructions

⚠️ IMPORTANT NOTES ⚠️

• All coordination efforts will be done in the "SN Validators" Telegram group.

• Make sure to backup your validator before making any changes.

• Please read carefully before you begin the upgrade.

⚠️ Cosmovisor tool may NOT be used with this upgrade. This is due to the following facts:

• Various prerequisites should be installed and configured

• After the installation, the secret node should perform an additional step, BEFORE the actual network upgrade (more on this later).

Upgrading Manually

There are 9 steps to be performed. Steps 1-4 inclusive can be performed BEFORE the upgrade height is reached (or before the upgrade is even proposed). We recommend doing so, i.e. perform those steps in advance, to minimize the downtime during the upgrade.

Technically the DCAP attestation isn't required. If your node is already registered on the network then the attestation during the upgrade isn't necessary, and we still support EPID attestation anwayay. But we strongly recommend making sure it works nevertheless. The EPID would be phased-out by Intel on April 2025.

1. Make sure you have the latest SGX DCAP driver

# 1. Make sure the SGX driver is installed. The following devices should appear:
# /dev/sgx_enclave
# /dev/sgx_provision

# If your kernel version if 5.11 or higher, then you probably already have the SGX driver installed.
# Otherwise - please update the kernel version.

# Also make sure that the user under which the node is supposed to run has privileges to access SGX
sudo groupadd sgx_prv
sudo usermod -a -G sgx_prv $USER

# Check if the above has effect, by the following command
groups

# The sgx_prv should appear. If it does not - Logout and re-login may be needed, for the change to take effect.

2. Install the DCAP runtime and AESM service

curl -fsSL https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key | sudo apt-key add -
. /etc/os-release; VERSION_CODENAME=${VERSION_CODENAME}
sudo add-apt-repository "deb https://download.01.org/intel-sgx/sgx_repo/ubuntu $VERSION_CODENAME main"
sudo apt-get update
sudo apt-get install -y \
    libsgx-aesm-launch-plugin \
    libsgx-enclave-common \
    libsgx-epid \
    libsgx-launch \
    libsgx-quote-ex \
    libsgx-uae-service \
    libsgx-qe3-logic \
    libsgx-pce-logic \
    libsgx-aesm-pce-plugin \
    libsgx-dcap-ql \
    libsgx-dcap-quote-verify \
    libsgx-urts \
    sgx-aesm-service \
    libsgx-aesm-ecdsa-plugin \
    libsgx-aesm-quote-ex-plugin \
    libsgx-ae-qve \
    libsgx-dcap-default-qpl	

sudo apt upgrade

If your system has 5th Gen Intel® Xeon® Scalable Processor(s)

For the DCAP attestation to work, you'll need to register your platform with Intel. This is achieved by the following:

sudo apt-get install -y sgx-ra-service

You can check the file /var/log/mpa_registration.log, to see if the platform is registered successfully.

3. Configure Quote Provider

The Quote Provider library is needed to provide the data for DCAP attestation. It should be installed by now (by the previous section), but before using it should be configured. The configuration file should be here:

/etc/sgx_default_qcnl.conf

If you're running a physical machine

The simplest would be to use the PCCS run by SecretLabs. Modify the following parameters in the file:

//PCCS server address
"pccs_url": "https://pccs.scrtlabs.com/sgx/certification/v4/"

You can set this parameter by the following command:

sudo cp /etc/sgx_default_qcnl.conf /etc/sgx_default_qcnl.conf.BKP
sudo sed -s -i 's/localhost:8081/pccs.scrtlabs.com/' /etc/sgx_default_qcnl.conf

Cloud computers

For cloud computers, the cloud service providers may provide their own PCCS. Please see their documentation

Note: You'll need to restart the AESMD service each time the configuration is changed

# Restart the AESMD service
sudo systemctl restart aesmd.service

4. Use check-hw to test the DCAP attestation (Optional step)

Download and run the check-hw tool (included in the Release package). You should see the following:

DCAP attestation ok
Platform verification successful! You are able to run a mainnet Secret node

That would mean all the above steps are ok, and you're good to go.

In case you see some error messages, but at the end the following:

Platform Okay!
Platform verification successful! You are able to run a mainnet Secret node

That would mean there's a problem with DCAP attestation. However the EPID attestation still works. Although you may technically run the node, it's strongly recommended to fix this. The EPID will be phased-out by Intel on April 2025.

To get a more detailed error info, run check-hw --testnet

5. Wait till the upgrade height is reached

When the network reaches the halt height you'll see this message in your node's log (journalctl -fu secret-node):

2:00PM ERR UPGRADE "v1.13" NEEDED at height: ...
2:00PM ERR CONSENSUS FAILURE!!! err="UPGRADE \"v1.13\" NEEDED at height: ...

Then, follow those steps to upgrade for v1.13: :warning: Note: The below instructions assume default installation. DO NOT COPY-PASTE if config.toml is not in ~/.secretd/config/config.toml or if you modified /etc/systemd/system/secret-node.service. If you have modified /etc/systemd/system/secret-node.service, you will need to re-apply those changes post installation and pre service restart.

6. Backup files, and install the new version

# Backup your node's SGX secrets
cp -a /opt/secret/.sgx_secrets ~/sgx_secrets_backup
mkdir 112backup
cp ~/.secretd/.node/* 112backup/


# Get the v1.13 binaries
wget "https://github.com/scrtlabs/SecretNetwork/releases/download/v1.13.0/secretnetwork_1.13.0_mainnet_goleveldb_amd64.deb"

# Stop the v1.12 node
sudo systemctl stop secret-node

# Install the v1.13 binaries
sudo apt install -y "./secretnetwork_1.13.0_mainnet_goleveldb_amd64.deb"

7. Migrate your local sealed files

This step must be done manually, before the Node automatic upgrade

# IMPORTANT: Upgrade sealed files to the newer SGX format
secretd migrate_sealing

8. Verify the DCAP attestation is working (Optional step)

This node version supports both EPID (older) and DCAP (newer) attestation schemes. Before proceeding, it's recommended to verify that DCAP attestation works properly. Though it's not mandatory if your node is already registered on the network.

# Retrieve DCAP-only attestation
secretd init-enclave --no-epid

The above retrieves self attestation. And since --no-epid flag is specified, that should be a DCAP attestation.

You can also verify that EPID attestation works, by the following command

# Retrieve EPID-only attestation
secretd init-enclave --no-epid

# Or retrieve any (EPID or DCAP or both) attestation
secretd init-enclave

9. Finally start the node

# re-apply any systemd unit file customizations

# Restart the node
sudo systemctl restart secret-node

After restarting the node with v1.13, you should see INF applying upgrade "v1.13" at height: ... in the logs (journalctl -fu secret-node). Once 67% of voting power comes online, you'll see blocks executing again.

Details of Upgrade Time

When the network reaches the halt height, the Secret Network blockchain will be halted and validators will need to take action to upgrade the chain to the secretd v1.13 binary (be it manually or automatically).

The upgrade is anticipated to take approx 30 minutes, during which time, there will not be any on-chain activity on the network.

In Case of an Upgrade Failure

In the event of an issue at upgrade time, we should coordinate via the "SN Validators" Telegram group.

If as a result of a software bug the network fails to produce new blocks with the v1.13 binaries, the SCRT Labs team will distribute a v1.12 binary with a replacement v1.13 upgrade handler, which will allow the chain to revert to v1.12 while continuing to produce new blocks.

Secret Network v1.5 Upgrade Instructions

⚠️ IMPORTANT NOTES ⚠️

• All coordination efforts will be done in the "SN Validators" Telegram group.

• Make sure to backup your validator before making any chnages.

• Please read carefully before you begin the upgrade.

Upgrading Manually

When the network reaches the halt height 5,941,700, you'll see this message in your node's log (journalctl -fu secret-node):

1:25PM ERR UPGRADE "v1.5" NEEDED at height: 5941700:
1:25PM ERR CONSENSUS FAILURE!!! err="UPGRADE \"v1.5\" NEEDED at height: 5941700

Then, the upgrade steps for v1.5 are:

⚠️ Note: if you have modified your systemd unit file (/etc/systemd/system/secret-node.service) you will need to re-apply those changes post installation and pre service restart.

# Stop the v1.4 node, to make sure that your process manager isn't trying to restart it while you upgrade
sudo systemctl stop secret-node

# Get & verify secretd v1.5

# Check what database type you're currently using:
awk -F \" '/^db_backend =/{print $2}' ~/.secretd/config/config.toml

# Uncomment the right binary based on the database type: `rocksdb` or `goleveldb`

## goleveldb
# wget "https://github.com/scrtlabs/SecretNetwork/releases/download/v1.5.0/secretnetwork_1.5.0_mainnet_goleveldb_amd64.deb"
# echo "483e25f6ba189d4763a81da94f32e20306370bb3111aa208d16c0141a4a0da7d secretnetwork_1.5.0_mainnet_goleveldb_amd64.deb" | sha256sum --check

## rocksdb
# wget "https://github.com/scrtlabs/SecretNetwork/releases/download/v1.5.0/secretnetwork_1.5.0_mainnet_rocksdb_amd64.deb"
# echo "4ff71dda2592c2d49fa7df723959ec51a5ba94890c0e804b63e95a38424607ae secretnetwork_1.5.0_mainnet_rocksdb_amd64.deb" | sha256sum --check

# Install v1.5 binaries
sudo apt install -y ./secretnetwork_1.5.0_mainnet_*_amd64.deb

# re-apply any systemd unit file customizations

# Restart the node
sudo systemctl restart secret-node

After restarting the node with v1.5, you should see INF applying upgrade "v1.5" at height: 5941700 in the logs (journalctl -fu secret-node). Once 67% of voting power comes online, you'll see blocks executing again.

Upgrading Automatically Using Cosmovisor

Cosmovisor is a new process manager for cosmos blockchains. It can make low-downtime upgrades smoother, as validators don't have to manually upgrade binaries during the upgrade, and instead can pre-install new binaries, and Cosmovisor will automatically update them based on on-chain SoftwareUpgrade proposals.

⚠️ Cosmovisor is still new and best practices for using it are not yet established. If you don't feel adventurous at this time, we recommend upgrading the manual way.

For instructions on how to setup Cosmovisor, go here.

Details of Upgrade Time

When the network reaches the halt height 5,941,700, the Secret Network blockchain will be halted and validators will need to take action to upgrade the chain to the secretd v1.5 binary (be it manually or automatically).

The proposal targets the upgrade proposal block to be 5,941,700, anticipated to be on Wednsday November 2, 2022 at 2:00PM UTC.

The upgrade is anticipated to take approx 30 minutes, during which time, there will not be any on-chain activity on the network.

In Case of an Upgrade Failure

In the event of an issue at upgrade time, we should coordinate via the "SN Validators" Telegram group.

If as a result of a software bug the network fails to produce new blocks with the v1.5 binaries, the SCRT Labs team will distribute a v1.4 binary with an empty v1.5 upgrade handler, which will essentially allow the chain to revert to v1.4 while continuing to produce new blocks.

Secret Network v1.4 "Shockwave Delta" Upgrade Instructions

⚠️ IMPORTANT NOTES ⚠️

• All coordination efforts will be done in the "SN Validators" Telegram group.

• Make sure to backup your validator before making any chnages.

• Please read carefully before you begin the upgrade.

Upgrading Manually

When the network reaches the halt height 5,309,200, you'll see this message in your node's log (journalctl -fu secret-node):

1:25PM ERR UPGRADE "v1.4" NEEDED at height: 5309200:
1:25PM ERR CONSENSUS FAILURE!!! err="UPGRADE \"v1.4\" NEEDED at height: 5309200

Then, the upgrade steps for v1.4 are:

⚠️ Note: if you have modified your systemd unit file (/etc/systemd/system/secret-node.service) you will need to re-apply those changes post installation and pre service restart.

# Stop the v1.3 node, to make sure that your process manager isn't trying to restart it while you upgrade
sudo systemctl stop secret-node

# Get & verify secretd v1.4

# Check what database type you're currently using:
awk -F \" '/^db_backend =/{print $2}' ~/.secretd/config/config.toml

# Uncomment the right binary based on the database type: `rocksdb` or `goleveldb`

## goleveldb
# wget "https://github.com/scrtlabs/SecretNetwork/releases/download/v1.4.0/secretnetwork_1.4.0_mainnet_goleveldb_amd64.deb"
# echo "f96ec8e04d6c7ad919d307435045c2bf538965d7786236a4ca7655f63caee77a secretnetwork_1.4.0_mainnet_goleveldb_amd64.deb" | sha256sum --check

## rocksdb
# wget "https://github.com/scrtlabs/SecretNetwork/releases/download/v1.4.0/secretnetwork_1.4.0_mainnet_rocksdb_amd64.deb"
# echo "42d203d1eec2d4d0720d3a9d206888bd7c1385c589fcc21a11514c95fcdbc169 secretnetwork_1.4.0_mainnet_rocksdb_amd64.deb" | sha256sum --check

# Install v1.4 binaries
sudo apt install -y ./secretnetwork_1.4.0_mainnet_*_amd64.deb

# re-apply any systemd unit file customizations

# Restart the node
sudo systemctl restart secret-node

After restarting the node with v1.4, you should see INF applying upgrade "v1.4" at height: 5309200 in the logs (journalctl -fu secret-node). Once 67% of voting power comes online, you'll see blocks executing again.

Upgrading Automatically Using Cosmovisor

Cosmovisor is a new process manager for cosmos blockchains. It can make low-downtime upgrades smoother, as validators don't have to manually upgrade binaries during the upgrade, and instead can pre-install new binaries, and Cosmovisor will automatically update them based on on-chain SoftwareUpgrade proposals.

⚠️ Cosmovisor is still new and best practices for using it are not yet established. If you don't feel adventurous at this time, we recommend upgrading the manual way.

For instructions on how to setup Cosmovisor, go here.

Details of Upgrade Time

When the network reaches the halt height 5,309,200, the Secret Network testnet blockchain will be halted and validators will need to take action to upgrade the chain to the secretd v1.4 binary (be it manually or automatically).

The proposal targets the upgrade proposal block to be 5,309,200, anticipated to be on Wednsday Septmber 21, 2022 at 2:00PM UTC.

The upgrade is anticipated to take approx 30 minutes, during which time, there will not be any on-chain activity on the network.

In Case of an Upgrade Failure

In the event of an issue at upgrade time, we should coordinate via the "SN Validators" Telegram group.

If as a result of a software bug the network fails to produce new blocks with the v1.4 binaries, the SCRT Labs team will distribute a v1.3 binary with an empty v1.4 upgrade handler, which will essentially allow the chain to revert to v1.3 while continuing to produce new blocks.

Secret Network v1.11 Instructions

⚠️ IMPORTANT NOTES ⚠️

• All coordination efforts will be done in the "SN Validators" Telegram group.

• Make sure to backup your validator before making any changes.

• The v1.11 upgrade will NOT include rocksdb binaries. We kindly ask anyone still running rocksdb to switch to goleveldb ASAP to avoid problems during the upgrade.

• Please read carefully before you begin the upgrade.

Upgrading Manually

Note: The below instructions assume default installation. DO NOT COPY-PASTE if config.toml is not in ~/.secretd/config/config.toml or if you modified /etc/systemd/system/secret-node.service. If you have modified /etc/systemd/system/secret-node.service, you will need to re-apply those changes post installation and pre service restart.

When the network reaches the halt height 10,824,000, you'll see this message in your node's log (journalctl -fu secret-node):

2:00PM ERR UPGRADE "v1.11" NEEDED at height: 10824000:
2:00PM ERR CONSENSUS FAILURE!!! err="UPGRADE \"v1.11\" NEEDED at height: 10824000

Then, the upgrade steps for v1.11 are:

# Backup your node's SGX secrets
cp -a /opt/secret/.sgx_secrets ~/sgx_secrets_backup
mkdir 110backup
cp ~/.secretd/.node/* 110backup/


# Get the v1.11 binaries
wget "https://github.com/scrtlabs/SecretNetwork/releases/download/v1.11.0/secretnetwork_1.11.0_mainnet_goleveldb_amd64.deb"

# Verify the v1.11 binaries
echo '60fcc57ff1cf1479255083cc4752de32ed776f059e9a11cde915277e379577a3 secretnetwork_1.11.0_mainnet_goleveldb_amd64.deb' |
    sha256sum --check

# Stop the v1.10 node
sudo systemctl stop secret-node

# Install the v1.11 binaries
sudo apt install -y "./secretnetwork_1.11.0_mainnet_goleveldb_amd64.deb"

# re-apply any systemd unit file customizations

# Restart the node
sudo systemctl restart secret-node

After restarting the node with v1.11, you should see INF applying upgrade "v1.11" at height: 10824000 in the logs (journalctl -fu secret-node). Once 67% of voting power comes online, you'll see blocks executing again.

Upgrading Automatically Using Cosmovisor

Cosmovisor is a new process manager for cosmos blockchains. It can make low-downtime upgrades smoother, as validators don't have to manually upgrade binaries during the upgrade, and instead can pre-install new binaries, and Cosmovisor will automatically update them based on on-chain SoftwareUpgrade proposals.

⚠️ Cosmovisor is still new and best practices for using it are not yet established. If you don't feel adventurous at this time, we recommend upgrading the manual way.

For instructions on how to setup Cosmovisor, go here.

Details of Upgrade Time

When the network reaches the halt height 10,824,000, the Secret Network blockchain will be halted and validators will need to take action to upgrade the chain to the secretd v1.11 binary (be it manually or automatically).

The proposal targets the upgrade proposal block to be 10,824,000, anticipated to be on Tuesday September 26, 2023 at 12:45PM UTC.

The upgrade is anticipated to take approx 30 minutes, during which time, there will not be any on-chain activity on the network.

In Case of an Upgrade Failure

In the event of an issue at upgrade time, we should coordinate via the "SN Validators" Telegram group.

If as a result of a software bug the network fails to produce new blocks with the v1.11 binaries, the SCRT Labs team will distribute a v1.10 binary with an empty v1.11 upgrade handler, which will allow the chain to revert to v1.10 while continuing to produce new blocks.

Secret Network v1.9 Instructions

⚠️ IMPORTANT NOTES ⚠️

• All coordination efforts will be done in the "SN Validators" Telegram group.

• Make sure to backup your validator before making any changes.

• Please read carefully before you begin the upgrade.

Upgrading Manually

Note: The below instructions assume default installation. DO NOT COPY-PASTE if config.toml is not in ~/.secretd/config/config.toml or if you modified /etc/systemd/system/secret-node.service. If you have modified /etc/systemd/system/secret-node.service, you will need to re-apply those changes post installation and pre service restart.

When the network reaches the halt height 8,861,800, you'll see this message in your node's log (journalctl -fu secret-node):

2:00PM ERR UPGRADE "v1.9" NEEDED at height: 8861800:
2:00PM ERR CONSENSUS FAILURE!!! err="UPGRADE \"v1.9\" NEEDED at height: 8861800

Then, the upgrade steps for v1.9 are:

# Backup your node's SGX secrets
cp -a /opt/secret/.sgx_secrets ~/sgx_secrets_backup
mkdir 18backup
cp ~/.secretd/.node/* 18backup/

# Figure out currently used db backend
DB_BACKEND=$(awk -F \" '/^db_backend =/{print $2}' ~/.secretd/config/config.toml)

# Get the v1.9 binaries
wget "https://github.com/scrtlabs/SecretNetwork/releases/download/v1.9.0/secretnetwork_1.9.0_mainnet_${DB_BACKEND}_amd64.deb"

# Verify the v1.9 binaries
echo '239ee0f4956a8dc6bbe17312861a67d12efa92bd03f653ed59fbac360e8aa676 secretnetwork_1.9.0_mainnet_goleveldb_amd64.deb
785a4863d7ab83ae83d30da07352de3542f492f5be45fdd3a7d34b960b5be44d secretnetwork_1.9.0_mainnet_rocksdb_amd64.deb' |
    grep "$DB_BACKEND" |
    sha256sum --check

# Stop the v1.8 node
sudo systemctl stop secret-node

# Install the v1.9 binaries
sudo apt install -y "./secretnetwork_1.9.0_mainnet_${DB_BACKEND}_amd64.deb"

# re-apply any systemd unit file customizations

# Restart the node
sudo systemctl restart secret-node

After restarting the node with v1.9, you should see INF applying upgrade "v1.9" at height: 8861800 in the logs (journalctl -fu secret-node). Once 67% of voting power comes online, you'll see blocks executing again.

Upgrading Automatically Using Cosmovisor

Cosmovisor is a new process manager for cosmos blockchains. It can make low-downtime upgrades smoother, as validators don't have to manually upgrade binaries during the upgrade, and instead can pre-install new binaries, and Cosmovisor will automatically update them based on on-chain SoftwareUpgrade proposals.

⚠️ Cosmovisor is still new and best practices for using it are not yet established. If you don't feel adventurous at this time, we recommend upgrading the manual way.

For instructions on how to setup Cosmovisor, go here.

Details of Upgrade Time

When the network reaches the halt height 8,861,800, the Secret Network blockchain will be halted and validators will need to take action to upgrade the chain to the secretd v1.9 binary (be it manually or automatically).

The proposal targets the upgrade proposal block to be 8,861,800, anticipated to be on Saturday May 16, 2023 at 2:00PM UTC.

The upgrade is anticipated to take approx 30 minutes, during which time, there will not be any on-chain activity on the network.

In Case of an Upgrade Failure

In the event of an issue at upgrade time, we should coordinate via the "SN Validators" Telegram group.

If as a result of a software bug the network fails to produce new blocks with the v1.9 binaries, the SCRT Labs team will distribute a v1.8 binary with an empty v1.9 upgrade handler, which will allow the chain to revert to v1.8 while continuing to produce new blocks.

Secret Network v1.7 Upgrade Instructions

⚠️ IMPORTANT NOTES ⚠️

• All coordination efforts will be done in the "SN Validators" Telegram group.

• Make sure to backup your validator before making any chnages.

• Please read carefully before you begin the upgrade.

Upgrading Manually

Note: The below instructions assume default installation. DO NOT COPY-PASTE if config.toml is not in ~/.secretd/config/config.toml or if you modified /etc/systemd/system/secret-node.service. If you have modified /etc/systemd/system/secret-node.service, you will need to re-apply those changes post installation and pre service restart.

When the network reaches the halt height 7,719,500, you'll see this message in your node's log (journalctl -fu secret-node):

2:00PM ERR UPGRADE "v1.7" NEEDED at height: 7719500:
2:00PM ERR CONSENSUS FAILURE!!! err="UPGRADE \"v1.7\" NEEDED at height: 7719500

Then, the upgrade steps for v1.7 are:

# Backup your node's SGX secrets
cp -a /opt/secret/.sgx_secrets ~/sgx_secrets_backup
cp ~/.secretd/.node/seed.json seed.json.backup

# Figure out currently used db backend
DB_BACKEND=$(awk -F \" '/^db_backend =/{print $2}' ~/.secretd/config/config.toml)

# Get the v1.7 binaries
wget "https://github.com/scrtlabs/SecretNetwork/releases/download/v1.7.1/secretnetwork_1.7.1_mainnet_${DB_BACKEND}_amd64.deb"

# Verify the v1.7 binaries
echo '32fa1dcc3e0d234f596d7efde917f9ca14df2292e6b6ef997eca20a435abcd95 secretnetwork_1.7.1_mainnet_goleveldb_amd64.deb
590573d0be9543440d2675ce7ae4e157bf44b4151fd38dca17575000bc8d2d44 secretnetwork_1.7.1_mainnet_rocksdb_amd64.deb' |
    grep "$DB_BACKEND" |
    sha256sum --check

# Stop the v1.6 node
sudo systemctl stop secret-node

# Install the v1.7 binaries
sudo apt install -y "./secretnetwork_1.7.1_mainnet_${DB_BACKEND}_amd64.deb"

# re-apply any systemd unit file customizations

# Restart the node
sudo systemctl restart secret-node

After restarting the node with v1.7, you should see INF applying upgrade "v1.7" at height: 6537300 in the logs (journalctl -fu secret-node). Once 67% of voting power comes online, you'll see blocks executing again.

Upgrading Automatically Using Cosmovisor

Cosmovisor is a new process manager for cosmos blockchains. It can make low-downtime upgrades smoother, as validators don't have to manually upgrade binaries during the upgrade, and instead can pre-install new binaries, and Cosmovisor will automatically update them based on on-chain SoftwareUpgrade proposals.

⚠️ Cosmovisor is still new and best practices for using it are not yet established. If you don't feel adventurous at this time, we recommend upgrading the manual way.

For instructions on how to setup Cosmovisor, go here.

Details of Upgrade Time

When the network reaches the halt height 7,719,500, the Secret Network blockchain will be halted and validators will need to take action to upgrade the chain to the secretd v1.7 binary (be it manually or automatically).

The proposal targets the upgrade proposal block to be 7,719,500, anticipated to be on Wednesday March 1, 2023 at 2:00PM UTC.

The upgrade is anticipated to take approx 30 minutes, during which time, there will not be any on-chain activity on the network.

In Case of an Upgrade Failure

In the event of an issue at upgrade time, we should coordinate via the "SN Validators" Telegram group.

If as a result of a software bug the network fails to produce new blocks with the v1.7 binaries, the SCRT Labs team will distribute a v1.6 binary with an empty v1.7 upgrade handler, which will essentially allow the chain to revert to v1.6 while continuing to produce new blocks.

Secret Network v1.12 Instructions

⚠️ IMPORTANT NOTES ⚠️

• All coordination efforts will be done in the "SN Validators" Telegram group.

• Make sure to backup your validator before making any changes.

• Please read carefully before you begin the upgrade.

Upgrading Manually

Note: The below instructions assume default installation. DO NOT COPY-PASTE if config.toml is not in ~/.secretd/config/config.toml or if you modified /etc/systemd/system/secret-node.service. If you have modified /etc/systemd/system/secret-node.service, you will need to re-apply those changes post installation and pre service restart.

When the network reaches the halt height 11,136,666, you'll see this message in your node's log (journalctl -fu secret-node):

2:00PM ERR UPGRADE "v1.12" NEEDED at height: 11136666:
2:00PM ERR CONSENSUS FAILURE!!! err="UPGRADE \"v1.12\" NEEDED at height: 11136666

Then, the upgrade steps for v1.12 are:

# Backup your node's SGX secrets
cp -a /opt/secret/.sgx_secrets ~/sgx_secrets_backup
mkdir 111backup
cp ~/.secretd/.node/* 111backup/


# Get the v1.12 binaries
wget "https://github.com/scrtlabs/SecretNetwork/releases/download/v1.12.0/secretnetwork_1.12.0_mainnet_goleveldb_amd64.deb"

# Verify the v1.12 binaries
echo '906589ad94a7bf8ec8bc5b54f6e3de7c2222e350cf1dbe61ac990fb95f1718b4 secretnetwork_1.12.0_mainnet_goleveldb_amd64.deb' |
    sha256sum --check

# Stop the v1.11 node
sudo systemctl stop secret-node

# Install the v1.12 binaries
sudo apt install -y "./secretnetwork_1.12.0_mainnet_goleveldb_amd64.deb"

# re-apply any systemd unit file customizations

# Restart the node
sudo systemctl restart secret-node

After restarting the node with v1.12, you should see INF applying upgrade "v1.12" at height: 11136666 in the logs (journalctl -fu secret-node). Once 67% of voting power comes online, you'll see blocks executing again.

Upgrading Automatically Using Cosmovisor

Cosmovisor is a new process manager for cosmos blockchains. It can make low-downtime upgrades smoother, as validators don't have to manually upgrade binaries during the upgrade, and instead can pre-install new binaries, and Cosmovisor will automatically update them based on on-chain SoftwareUpgrade proposals.

⚠️ Cosmovisor is still new and best practices for using it are not yet established. If you don't feel adventurous at this time, we recommend upgrading the manual way.

For instructions on how to setup Cosmovisor, go here.

Details of Upgrade Time

When the network reaches the halt height 11,136,666, the Secret Network blockchain will be halted and validators will need to take action to upgrade the chain to the secretd v1.12 binary (be it manually or automatically).

The proposal targets the upgrade proposal block to be 11,136,666, anticipated to be on Tuesday October 17, 2023 at 2:00 PM UTC.

The upgrade is anticipated to take approx 30 minutes, during which time, there will not be any on-chain activity on the network.

In Case of an Upgrade Failure

In the event of an issue at upgrade time, we should coordinate via the "SN Validators" Telegram group.

If as a result of a software bug the network fails to produce new blocks with the v1.12 binaries, the SCRT Labs team will distribute a v1.11 binary with a replacement v1.12 upgrade handler, which will allow the chain to revert to v1.11 while continuing to produce new blocks.

Secret Network v1.3 "Shockwave Alpha" Network Upgrade Instructions

⚠️ IMPORTANT NOTES ⚠️

• All coordination efforts will be done in the "SN Validators" Telegram group.

• Make sure to before making any chnages.

• Please read carefully before you begin the upgrade.

Table of Contents

Upgrading Manually

When the network reaches the halt height 3,343,000, you'll see this message in your node's log (journalctl -fu secret-node):

Then, the upgrade steps for v1.3 are:

⚠️ Note: uncomment the right binary based on the database type on the node that you're upgrading: rocksdb vs. goleveldb.

After restarting the node with v1.3, you should see INF applying upgrade "v1.3" at height: 3343000 in the logs (journalctl -fu secret-node). Once 67% of voting power comes online, you'll see blocks executing again.

Upgrading Automatically Using Cosmovisor

Cosmovisor is a new process manager for cosmos blockchains. It can make low-downtime upgrades smoother, as validators don't have to manually upgrade binaries during the upgrade, and instead can pre-install new binaries, and Cosmovisor will automatically update them based on on-chain SoftwareUpgrade proposals.

Details of Upgrade Time

When the network reaches the halt height 3343000, the Secret Network blockchain will be halted and validators will need to take action to upgrade the chain to the secretd v1.3 binary (be it manually or automatically).

The upgrade is anticipated to take approx 30 minutes, during which time, there will not be any on-chain activity on the network.

In Case of an Upgrade Failure

In the event of an issue at upgrade time, we should coordinate via the "SN Validators" Telegram group.

If as a result of a software bug the network fails to produce new blocks with the v1.3 binaries, the SCRT Labs team will distribute a v1.2 binary with an empty v1.3 upgrade handler, which will essentially allow the chain to revert to v1.2 while continuing to produce new blocks.

Setting Up A Node Using Cosmovisor

Cosmovisor is a new process manager for cosmos blockchains. It can make low-downtime upgrades smoother, as validators don't have to manually upgrade binaries during the upgrade, and instead can pre-install new binaries, and Cosmovisor will automatically update them based on on-chain SoftwareUpgrade proposals.

You should review the docs for Cosmovisor located here:

Install Cosmovisor

Set Up A New v1.2 Node

Now the Cosmovisor directory structure should look like this:

After making the proper configurations to your new node, launch it using sudo systemctl start cosmovisor. You should now see blocks executing in the logs (journalctl -fu cosmovisor).

Migrate A Running v1.2 Node

Now the Cosmovisor directory structure should look like this:

To relaunch your node, use sudo systemctl start cosmovisor. You should now see blocks executing in the logs (journalctl -fu cosmovisor).

Prepare A v1.3 Node Upgrade (Shockwave Alpha)

The "Shockwave Alpha" upgrade is anticipated to be on Wednesday April 27th, 2022 at 2:00PM UTC. Below are instructions to prepare Cosmovisor for automatically upgrading your node to v1.3.

Now the Cosmovisor directory structure should look like this:

If set up properly, when the time comes Cosmovisor will perform an automatic v1.2 to v1.3 upgrade.

Secret Network v1.4 "Shockwave Delta" Testnet Upgrade Instructions

Upgrading Manually

When the network reaches the halt height 4,711,950, you'll see this message in your node's log (journalctl -fu secret-node):

Then, the upgrade steps for v1.4 are:

⚠️ Note: if you have modified your systemd unit file (/etc/systemd/system/secret-node.service) you will need to re-apply those changes post installation and pre service restart.

⚠️ Note: uncomment the right binary based on the database type you're using: rocksdb vs goleveldb. To check what database type you're currently using: awk -F \" '/^db_backend =/{print $2}' ~/.secretd/config/config.toml.

After restarting the node with v1.4, you should see INF applying upgrade "v1.4-fix" at height: 4711950 in the logs (journalctl -fu secret-node). Once 67% of voting power comes online, you'll see blocks executing again.

Upgrading Automatically Using Cosmovisor

Cosmovisor is a new process manager for cosmos blockchains. It can make low-downtime upgrades smoother, as validators don't have to manually upgrade binaries during the upgrade, and instead can pre-install new binaries, and Cosmovisor will automatically update them based on on-chain SoftwareUpgrade proposals.

Details of Upgrade Time

When the network reaches the halt height 4,711,950, the Secret Network testnet blockchain will be halted and validators will need to take action to upgrade the chain to the secretd v1.4 binary (be it manually or automatically).

The proposal targets the upgrade proposal block to be 4,711,950, anticipated to be on Sunday Septmber 11, 2022 at 2:00PM UTC.

The upgrade is anticipated to take approx 30 minutes, during which time, there will not be any on-chain activity on the network.

In Case of an Upgrade Failure

In the event of an issue at upgrade time, we should coordinate via the "SN Testnet Validators" Telegram group.

If as a result of a software bug the network fails to produce new blocks with the v1.4 binaries, the SCRT Labs team will distribute a v1.3 binary with an empty v1.4 upgrade handler, which will essentially allow the chain to revert to v1.3 while continuing to produce new blocks.

Network Upgrade Instructions From secret-1 To secret-2

⚠️ Please read carefully before you begin the upgrade.

• Network Upgrade Instructions from secret-1 to secret-2

• Validators

  • Prepare your secret-1 validator to halt after block #1,246,400

  • Install the new binaries on your SGX machine

  • Migrate your validator's signing key

  • Migrate your validator's wallet

  • Set up your SGX machine and become a secret-2 validator

• In case of an upgrade failure

Validators

All coordination efforts will be done in the channel in the Secret Network Discord.

⚠️ Don't delete your secret-1 machine, as we might have to relaunch it.

⚠️ If your SGX machine has a previous secretnetwork installation on it (e.g. from the testnet), you will need to remove it before you continue:

Prepare Your secret-1 Validator To Halt After Block #1,246,400

On the old machine (secret-1):

2. Install The New Binaries On Your SGX Machine

On the new SGX machine (secret-2):

3. Migrate Your Validator's Signing Key

Copy your ~/.secretd/config/priv_validator_key.json from the old machine (secret-1) to the new SGX machine (secret-2) at the same location.

4. Migrate Your Validator's Wallet

Export the self-delegator wallet from the old machine (secret-1) and import to the new SGX machine (secret-2).

On the old machine (secret-1) use secretcli keys export "$YOUR_KEY_NAME". On the new SGX machine (secret-2) use secretcli keys import "$YOUR_KEY_NAME" "$FROM_FILE_NAME"

Notes:

1. If you're recovering the wallet using secretcli keys add "$YOUR_KEY_NAME" --recover you should also use --hd-path "44'/118'/0'/0/0".

2. If the wallet is stored on a Ledger device, use --legacy-hd-path when importing it with secretcli keys add.

5. Set Up Your SGX Machine And Become A secret-2 Validator

On the new SGX machine (secret-2):

Now wait until you're done catching up. This is fast. Once the following command outputs true you can continue:

Once your node is done catching up, you can unjail your validator:

You’re now a validator in secret-2! 🎉

To make sure your validator is unjailed, look for it in here:

In Case Of An Upgrade Failure

If after a few hours the Enigma team announces on the chat that the upgrade failed, we will relaunch secret-1.

1. On the old machine (secret-1):
2. Wait for 67% of voting power to come back online.