Ultimate Spacemesh Setup - Nodes! (5/7)

All your hard work is about to pay off

Github: Hake Hardware

It’s finally time. We have Ubuntu + Docker set up. Portainer to manage our containers. Prometheus, Node Exporter, and cAdvisor set up for monitoring. And now we will get our nodes running and publishing metrics to Prometheus.

Folder Structure

Our nodes will have bind mounts to two folders:

1. nodes/nodeXX

2. /media/postdataXX

Since I will have 15 nodes eventually, the ‘X’ above represent the node number, so ‘node01’ for node 1 and ‘postdata01’ for postdata for node 1. You can really name these folders whatever you want, but just make sure to change them everywhere as they are referenced often.

First thing I will do is create a parent folder for all of my ‘node’ folders.

mkdir ~/nodes && cd ~/nodes

This will create the folder and then move into the folder. From here you need to create a folder for each node.

mkdir node01 node02 node03

Here I am making 3 node folders. You can add more to suit your needs. Next we need to create the config file for each node. Each config file will be different because we will be using different metric and TCP ports for each node. My nodes also have a different amount of Space Units, so I’ll need to modify the numUnits field for each config. I’ve found the best way is to download the config file from the official source here: Config

Then modify it to your needs and copy and paste it into the node folder. So to do this you can do the following. Move into the node folder:

cd node01

Create the config.mainnet.json file:

nano config.mainnet.json

Then paste in your config. You can find mine on github here: config.mainnet.json

A few things to node:

1. In the ‘p2p’ section, the tcp port should be different for each node. So something like:

   1. node01: "listen": "/ip4/0.0.0.0/tcp/6001"

   2. node02: "listen": "/ip4/0.0.0.0/tcp/6002"

2. The same thing goes for the metrics-port in the ‘main’ section:

   1. node01: "metrics-port": 10001

   2. node02: "metrics-port": 10002

3. In the ‘smeshing’ section update the numUnits to match your postdata

   1. "smeshing-opts-numunits": 6

4. If your postdata is not complete, then make sure ‘smeshing-start’ is false

   1. "smeshing-start": false

5. Enter your own coinbase:

   1. "smeshing-coinbase": "<your coinbase>"

6. You should update your nonces and threads to what is appropriate for your hardware and postdata size. I will not be covering this in this guide.

Repeat this process for each node directory. After that, create the sm_data folder in each node directory:

mkdir sm_data

The architecture looks something like this, where you have created the parts where the red line is.

The node will automatically create the files in the ‘sm_data’ folder. If you already have a copy of the state, you can copy it into the ‘sm_data’ folder to reduce sync time.

Mount Drives w/ PoST Data

If you are starting with fresh HDDs and have not created postdata yet, skip to the next section.

If your postdata is on an separate HDD from your root file system then you will need to mount your HDD. If you are using a USB drive it may already be mounted. You can check this by running:

df -h

If you see your drive already mounted, you will need to unmount it and remount it. Alternately you can update the config file with your mount location. If you want to umount the drive you can run the following, and replace /dev/sdX with your drive. You might see /dev/nvme0n1 or something if you have an NVME

sudo umount /dev/sdX

We will be mounting all of our drives the /media directory. Go ahead and make a mount point for each HDD you have:

sudo mkdir /media/postdata01

For each mount point increase the number, so postdata02 for HDD 2. Next, get the UUID of the drive:

lsblk -f

Copy the UUID and then update your fstab:

sudo nano /etc/fstab

Then at the bottom enter:

UUID=<your uuid>       /media/postdata01 ext4    defaults,noatime,nofail 0       0

You will want a new entry for each HDD, making sure to update the UUID and mount point for each drive. Mine looks like this:

Press CTRL+X then ‘Y’ to save and exit. The mount the drives with:

sudo mount -av

This will mount the drives. In the future the drives will automatically mount when you start the host. You should all see the drives when you run:

df -h

Mount Drives - No PoST Data

If you are starting with fresh drives, first identify the drive with:

lsblk

Then format with ext4. Replace the ‘sdX’ with whatever your drive is. If it is an NVME drive you might see /dev/nvme0n0 or something.

sudo mkfs -t ext4 /dev/sdX

Then remove any reserved space:

sudo tune2fs -r 0 /dev/sdX

Repeat this for each drive you have. When done, create the mount points. For each HDD you will want to create a mount point:

sudo mkdir /media/postdata01

Run this again for each drive, but increase the postdata number each time, so postdata02 for the second, and so on. When you are done get the UUID for all your drives with:

lsblk -f

And then update fstab:

sudo nano /etc/fstab

And then for each drive create an entry:

UUID=<your uuid>       /media/postdata01 ext4    defaults,noatime,nofail 0       0

You will want a new entry for each HDD, making sure to update the UUID and mount point for each drive. Mine looks like this:

At this point, all of your drives should be mounted, and your node folders created, and your configs added. Something like this:

Where the red lines are the things you should have created. The top represents the HDDs getting mounted to those mount points. The bottom is the folder structure you created.

Stacks on Stacks

Its time to add our go-spacemesh containers. We will do this in Portainer, so bring that up and go to the ‘Stacks’ Editor again. Open up my github and copy the content: stack

Then paste this into your stack, replacing everything (unless you made customization). If you made customization to the stack, you will need to check out the github link and then make the appropriate adjustments.

Now when it’s been updated, click ‘Deploy the stack’ and wait for the stack to deploy. All the nodes should now be running! But in order be sure we need to get the metrics going. Go back to your shell and open up the ‘prometheus.yml’ file again:

sudo nano /etc/prometheus/prometheus.yml

Just like before, check out my github and copy and paste the final code: prometheus

Save that and then go back to Portainer and restart prometheus. Now open up Prometheus in your browser and go to ‘Targets’ It will take quite some time for the nodes to show their State as ‘Up’. I’m talking as long as 30 minutes. It really depends on your hardware and if you are syncing from scratch. Be patient and give it some time. And that’s it! You now have your nodes running in Docker. Oh wait!! We need to set up your Spacemesh Dashboard. Okay this one is a bit tougher because there an ID you can just enter. You need to actually paste do a bit of work.

Spacemesh Dashboard

Alright.. so the first thing you need to do is open up Grafana. Go to ‘Dashboards’ then click ‘New’. Click ‘Import’ like before. But instead of entering in an ID, you need to paste in a JSON. The JSON is located.. on my github: Spacemesh Dash

So paste it in there and then click ‘Load’. Update the name of the dashboard to whatever you want. Then click ‘Import’. It won’t work. You are going to need to update the data source. Click the hamburger icon, then go to ‘Connections’. Then click ‘Data Sources’

Then select your data source that you added earlier. I have 3 hosts so mine shows 3 data sources, but you should only have one. Select it.

Now there might be a better way to get this, but what we need is the UID for your data source. It can be found in the URL after you select your data source. So go ahead and copy it. Note that it is everything after the ‘/’

Copy that and save it somewhere. Then go back to dashboards, and open up the dashboard we just imported. There is a gear icon, click it.

Then on the left side, click ‘JSON Model’

Okay this part is hard to explain. But what you want to do is click into the section with the JSON and press ‘CTRL+F’. This should open up the search box in the top right. Click the little carrot to expand it so two inputs are showing like in the photo below. Then find the ‘uid’ (underlined in the photo’ and copy just the part in orange and paste it into the top box of search input. Next grab the UID we copied earlier and paste it into the bottom input. What we are doing is finding and replacing the existing UID with your data source UID. This way the data can load. Once both values are in the boxes, you can replace all of the UIDs, I believe there are 19 entries. Replace them all.

Now click “Save Changes’ an “Save Dashboard”. Once the nodes are online and you can see them in your Prometheus Targets, then you should see them in your dashboard. As I mentioned before it may take some time. And that is it! Your nodes are running, the metrics dashboard is going, and all you need to do is sit back and relax. Okay you probably have a lot of questions still. But that is what the next section is for! So I’ll see you there.