KMS_YubiHSM Set up l Cosmos Hub
Signing on Cosmos Testnet using tmkms + YubiHSM
1. What is KMS (Key Management System)?
In simple terms, KMS is a service which manages keys. In Cosmos Hub, which uses the Tendermint Consensus Algorithm, we can use Tendermint KMS (tmkms) to manage keys.
2. What is Tendermint KMS (tmkms)?
Tmkms is a KMS which is built with Rust, and can be used on blockchain applications which uses Tendermint Consensus, such as Cosmos Hub, Terra and IRISnet. At the time of the writing of this article, KMS is a beta version at v0.6.3. It is an open source software which follows “Apache License, Version 2.0”, so anyone can freely use it.
By using Tendermint KMS, we can benefit from the following features:
- High Availability (H/A) on validator signing key
- Double Sign protection
- HSM (Hardware Security Module) usage
You can find more details on Tendermint KMS on the Github page.
3. HSM (Hardware Security Module)
HSM could be a physical device or a cloud service which provides encrypting, deciphering, signing and validating features and protects digital keys. We can use it to safely secure validator’s private key (priv_validator_key.json) on blockchain applications which use Tendermint Consensus.
Currently, Tendermint KMS supports below HSM:
Now, let’s use Tendermint KMS and YubiHSM2 to sign on Cosmos Testnet (gaia-13005)
The testing environment is as below:
- Node Server: AWS EC2, ubuntu 18.04
- KMS Server: Physical Server (Dell), ubuntu 18.04
- KMS: tmkms v0.6.0-rc0
- HSM: YubiHSM2
- Tendermint: v.0.23.2
- Gaiad/Gaiacli: v1.0.0
- Cosmos Testnet: gaia-13005
We will assume that all node configuration and block sync is completed on Cosmos Testnet, and we will proceed to the procedure below:
Test#1. Cosmos Testnet Node Setting (Skip)
Test#2. Reset YubiHSM2
Test#3. Add new user to KMS Server
Test#4. Install libusb, Rust, pkg-config, tmkms on KMS Server
Test#5. Create tmkms configuration file (tmkms.toml) on KMS Server
Test#6. Add validator private key (priv_validator_key.json) to YubiHSM2
Test#7. Run tmkms
Test#8. Change config (config.toml) on Cosmos Node
Test#9. Restart Cosmos Node
Test#10. Confirm sign
Test#11. Register tmkms.service on KMS Server
Test#1. Cosmos Testnet Node Setting (Skip)
“ATEAM” node is running well on gaia-13005 at this moment.
Cosmos Testnet Explorer: https://gaia-13k5.bigdipper.live
Test#2. Reset YubiHSM2
When you eject and insert YubiHSM2 while pressing the tip of the device for more than 3 seconds, YubiHSM2 resets itself and the blinking LED pattern changes. (The video shows the same YubiHSM2, but there are cases when the LED pattern changes very visibly, and there are cases when the pattern does not change visibly).
※ Same applies to Windows PC, and when you look at the Device Manager (devmgmt.msc), you will see that YubiHSM device reappear on the screen.
Test#3. Add new use to KMS Server
After creating a new user, we have to add new settings to udev in order to let the user access YubiHSM2.
Udev is a device manager for Linux Kernel which manages devices for /dev which handles all user space actions when adding or removing devices. (Source: https://wiki.gentoo.org/wiki/Udev/en)
While accessing YubiHSM2 without adding a new setting to udev, with an account which is not a root, we faced the error below.
error: error connecting to YubiHSM2: protocol error: USB error: USB(bus=1,addr=22): error opening device: Access denied (insufficient permissions))
We were informed that by using the command below after applying udev is the right way, but since the command did not work, we reboot the server and tried once again.
#udevadm control-reload-rules&&udevadm trigger
// proceed with root account
# mkdir /data_tmkms
# useradd -m -d /data_tmkms/tmkms -G sudo tmkms -s /bin/bash// Add settings to udev
# vi /etc/udev/rules.d/10-yubihsm.rules
...
SUBSYSTEMS=="usb", ATTRS{product}=="YubiHSM", GROUP=="tmkms"
...// Reboot server
# reboot
Test#4. Install libusb, rust, pkg-config, tmkms on KMS Server
// Proceed with tmkms account// Install libusb
$ sudo apt install libusb-1.0-0-dev// install rust
// Choose "1) Proceed with installation(default)"
$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
$ source $HOME/.cargo/env
$ cargo --version// Install pkg-config
$ sudo apt install pkg-config -y// Install tmkms(v0.6.3)
$ cd $HOME
$ git clone https://github.com/tendermint/kms.git
$ cd $HOME/kms
$ cargo install tmkms --features=yubihsm --version=0.6.3
$ tmkms version
Test#5. Create tmkms configuration file (tmkms.toml) on KMS Server
Example of tmkms.toml:
https://github.com/tendermint/kms/blob/master/tmkms.toml.example
Input validator information on “addr” part of [[validator]], and you can set Port freely but it must match the set Port of “priv_validator_laddr” which was set on config.toml of the validator.
For more information about Validator settings, please refer to [Test#8. Change config (config.toml) on Cosmos Node]
$ mkdir $HOME/kms/cosmos
$ cd $HOME/kms/cosmos
$ vi tmkms.toml
...
[[chain]]
id = "gaia-13005"
key_format = { type = "bech32", account_key_prefix = "cosmospub", consensus_key_prefix = "cosmosvalconspub" }
state_file = "/data_tmkms/tmkms/kms/cosmos/cosmoshub_priv_validator_state.json"
# state_hook = { cmd = ["/path/to/block/height_script", "--example-arg", "cosmoshub"] }## Validator configuration
[[validator]]
addr = "tcp://49.247.215.16:46658"
# or addr = "unix:///path/to/socket"
chain_id = "gaia-13005"
reconnect = true # true is the default
secret_key = "/data_tmkms/tmkms/kms/cosmos/secret_connection.key"## Signing provider configuration
# enable the `yubihsm` feature to use this backend
[[providers.yubihsm]]
adapter = { type = "usb" }
#auth = { key = 1, password_file = "/path/to/password" } # or pass raw password as `password`
auth = { key = 1, password = "password" }
keys = [{ chain_ids = ["gaia-13005"], key = 1 }]
...
Test#6. Add validator private key (priv_validator_key_json) to YubiHSM2
Take private key (priv_validator_key.json) from the validator node and input to YubiHSM2
$ cd $HOME/cosmos// Take private key from validator node
$ scp cosmos@49.247.215.16:/data_cosmos/cosmos/.gaiad/config/priv_validator_key.json ./// Add to YubiHSM2
$ tmkms yubihsm keys import -t json -i 1 priv_validator_key.json// Confirm that the key is added to YubiHSM2
// 결과에서 나오는 주소는 Cosmos Node에서 "gaiad tendermint show-validator" 명령어를 통해 확인할 수 있는 주소와 같다(Consensus public address)
& tmkms yubihsm keys list
Explanation on “tmkms yubihsm keys import -t json -i 1 priv_validator_key.json”
-t option: Set key type (We are going to add a json file, so set it as “json”)
-i option: Key ID (Set Key ID as the one set in [[providers.yubihsm]] of tmkms.toml config)
Test#7. Run tmkms
An error occurs when you run tmkms because the validator node is not ready yet
$ tmkms start -c $HOME/kms/cosmos/tmkms.toml
-c option: Set config file (tmkms.toml) and run
Test#8. Change config (config.toml) on Cosmos Node
Comment out the parts related to “priv_validator_key_file” of config.toml from gaiad, and input validator’s private IP and Port (default: 26658) on “priv_validator_laddr”
$ vi $HOME/.gaiad/config/config.toml
...
#priv_validator_key_file = "config/priv_validator_key.json"
priv_validator_laddr = "tcp://10.4.0.4:46658"
...When we set validator’s public IP, localhost, 127.0.0.1 on priv_validator_laddr, an Error occured saying (ERROR: error with private validator socket client: failed to start private validator: accept tcp 127.0.0.1:46658:i/o timeout). This was fixed when we input the validator’s private IP
Test#9. Restart Cosmos Node
To apply changes made on config.toml on Test#8, we must restart gaiad
// if gaiad is registered as a service
sudo systemctl restart gaiad.serviceTest#10. Confirm sign
When Cosmos validator node and tmkms is connected, Prevote and Precommit information appears on tmkms
Test#11. Register tmkms.service on KMS Server
$ sudo vi /etc/systemd/system/tmkms.service
...
[Unit]
Description=Cosmos tmkms
After=network.target[Service]
User=tmkms
WorkingDirectory=/data_tmkms/tmkms/.cargo/bin
ExecStart=/data_tmkms/tmkms/.cargo/bin/tmkms start -c /data_tmkms/tmkms/kms/cosmos/tmkms.toml
Restart=always
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=tmkms
RestartSec=3
LimitNOFILE=4096[Install]
WantedBy=multi-user.target
...$ sudo systemctl enable tmkms.service
$ sudo systemctl start tmkms.service
The log is recorded at /var/log/syslog, and to check the log in real-time, you may input:
sudo tail -f /var/log/syslog
※ ETC
During the test, we used the validator’s public IP to connect validator node and tmkms, but when you are using the procedure above, we recommend that you use a private IP address :)
※ References
Node A-Team (Moniker: ATEAM) is a Cosmos-SDK and Tendermint-based Blockchain Validator, which currently participates in Cosmos, IRISnet and Terra. Based on the knowledge and experience gained from various testnets and mainnet, Node A-Team contributes to Cosmos Ecosystem and operates highly secure nodes.
#ATEAM History
Cosmos: “Game of Stakes” — Never Jailed
IRISnet: “FUXI Betanet”, “Nyancat Testnet”— Reward Winner
Terra-project: “Genesis Drill” — Top Tier
#ATEAM Info
[ATEAM Validator Address: cosmosvaloper14l0fp639yudfl46zauvv8rkzjgd4u0zk2aseys]
Webpage: https://nodeateam.com/
Twitter: https://twitter.com/Node_Ateam
E-mail: contact@nodeateam.com
