Network Migrations

This document provides a detailed guide on how to establish a new network that integrates existing data from a previous network, using nodes that run on the updated version of the kwild binaries. This approach is necessary when seamless upgrades to new versions are not feasible due to significant, breaking changes that affect consensus mechanisms and transaction replays.

Network Migrations

Follow the steps below to upgrade the network to a new version of the kwild binaries:

Create Snapshot

Stop the network by shutting down all the nodes in the network. Then, use the kwil-admin snapshot tool to take a snapshot of the final state of the Kwild database. For more details, refer to the create database snapshots documentation. This tool connects directly to the database to capture its state and does not require the kwild process to be running.

The kwil-admin snapshot tool generates a snapshot file and a genesis file required to start the new network. The genesis file includes the genesis app hash, derived from the hash of the uncompressed snapshot file, and the genesis validators set to the validators from the previous network. To start a new network with the snapshot data, the genesis file must have the app_hash field set; otherwise, the network will start with an empty state.

kwil-admin snapshot create --snapdir /path/to/snapdir

# Running the above command will generate the snapshot file and the genesis.json file in the snapshot directory.
ls /path/to/snapdir
genesis.json            kwildb-snapshot.sql.gz

By default, the above command connects to the postgres instance runnning at localhost:5432 with the default user postgres with trust authentication mode. To connect to a database running on a different host/port/user/password, use the following command:

kwil-admin snapshot create --snapdir /path/to/snapdir --dbname <dbname> --host <dbhost> --port <dbport> --user <dbuser> --password <dbpass>

Verify App Hash

The app hash set in the genesis file is generated based on the hash of the uncompressed snapshot. You can verify the app hash in the generated genesis file as follows

# Uncompress the snapshot file.
gunzip /path/to/snapdir/kwildb-snapshot.sql.gz

# Generate app hash using sha256sum.
shasum -a 256 /path/to/snapdir/kwildb-snapshot.sql
4c2ecfc145c37a45a1fd21cd38ee84db80d852996fc30438a007ba624387c683  /path/to/snapdir/kwildb-snapshot.sql

# app_hash in genesis file is a hexadecimal representation of the sha256sum hash encoded in base64.
# hex-> binary -> base64
echo -n "4c2ecfc145c37a45a1fd21cd38ee84db80d852996fc30438a007ba624387c683" | xxd -r -p | base64

TC7PwUXDekWh/SHNOO6E24DYUplvwwQ4oAe6YkOHxoM=

# compare the app hash with the genesis file
cat /path/to/snapdir/genesis.json | jq .app_hash

"TC7PwUXDekWh/SHNOO6E24DYUplvwwQ4oAe6YkOHxoM="

Distribute Genesis State

Distribute the snapshot and the genesis.json files required to start the new network.

Ensure that the genesis.json file has the app_hash field set to the hash of the uncompressed snapshot file. Refer to the Verify App Hash section to ensure the hash is correct. If any other changes are needed to the genesis configuration, update the genesis file manually before distributing it.

Run Genesis Nodes

Install Kwild Binaries

Install the new version of the kwild binaries. Refer to the Installation documentation for more details on how to install the kwild binaries.

Configure Node to Use Snapshot

Setup the files required to start a new node. Refer to the Node Configuration documentation or use the kwil-admin tool:

kwil-admin setup init -o /path/to/node/root/dir

Next, copy the snapshot, genesis.json, and the original validator's private_key files into the new node's root directory.

note

The private_key needs to be copied because the new network will use the same validator set as the previous network.

# Copy the snapshot and genesis files to the node root directory.
cp /path/to/snapdir/genesis.json /path/to/node/root/dir
cp /path/to/snapdir/kwildb-snapshot.sql /path/to/node/root/dir

# Copy the old validator's private key to the node root directory.
cp /path/to/private_key /path/to/node/root/dir

The node root directory should have the following structure:

root_dir
├── config.toml
├── private_key
├── genesis.json
├── kwildb-snapshot.sql
├── abci/

To initialize the node with the snapshot data, set the genesis_state path in config.toml to the location of the downloaded snapshot file. The node will fail to start if the genesis_state path is not provided in the config.toml.

#######################################################################
###                      App Config Options                         ###
#######################################################################

[app]

# Path to the snapshot file to restore the database from.
# Used during the network migration process.
genesis_state = "/path/to/snapshot/file"

Start the Node

Once the genesis and config files are configured to use snapshots, start the node using the following command:

# use -h for more options. 
kwild -r /path/to/node/root/dir

tip

If your genesis.json file has multiple validators, you will need to start >2/3 of the validators before the network can start producing blocks.

Nodes Catching Up

Follow the same steps described in the Run Genesis Nodes section to configure and join the network. Nodes can either sync with the network using statesync or blocksync.

If the node uses blocksync, the genesis_state path is mandatory to ensure that the node initializes with the same genesis initial state and replay all the historical blocks. If the node doesn't start from the same initial state, it will eventually fork itself off the chain during blocksync.

If the node uses statesync to catch up with the network, the genesis_state path is not required in the config.toml file. The node will use the statesync protocol to catch up with the network as described in the State Sync documentation.