Guard your secrets in the “Vault”

Written by Mohamed Elzomor, Software Engineer II in the Couch Potatoes squad at Kites Software

There is always a bunch of secrets in your application that you want to keep secret, such as database credentials, secret keys for encoding/decoding, API keys.. etc. The question now is how much are you keeping your secrets secret?

One of the most used ways is to make a .env file and keep it untracked from Git, and then create a .env.example tracked file so that you can make it easier to re-use on the server or locally for development. One drawback for this method is that you need to ssh on the server, so that you can update the .env file, so each time you have to go to the server and login to it. If the owner of the ssh key is not available, you might have to generate a new ssh, spreading the access to the server, which is totally a bad idea.

Another way, is to keep your secrets away in a reachable place, like AWS secrets manager. However, this comes with a cost (Literal cost). However, Hashicorp has introduced its Vault for free with a bit limited functionalities but they will be more than enough for your application, if you just need a place to set secrets, update or even delete them via a super user-friendly UI. shall we dive in?

Build your Vault

1. Create directory that contains the following: (docker-compose.yml, vault/config/vault.hcl, valult/data)
2. In the docker compose file, we will have this (If you run in the dev mode, the database will not be persistent)

services:
  vault:
    image: hashicorp/vault:1.17
    container_name: vault
    ports:
      - "8200:8200"
    volumes:
      - ./vault/config/vault.hcl:/vault/config/vault.hcl
      - ./vault/data:/vault/data
    cap_add:
      - IPC_LOCK
    environment:
      VAULT_ADDR: http://127.0.0.1:8200
    command: server # To run in production mode, if in dev, just remove the line

3. The settings of the Vault, you can find here (vault/config/vault.hcl)

storage "raft" {
  path = "/vault/data"
  node_id = "raft_node_1"
}

listener "tcp" {
  address = "0.0.0.0:8200"
  tls_disable = "true"
}

ui = true
cluster_addr = "http://127.0.0.1:8201"
api_addr = "http://127.0.0.1:8200"

Here, we are using raft as our database(As suggested by Hashicorp docs)

After we complete the previous steps, we are ready to add secrets to our application once we run this command

docker compose up -d --build

Initialize your Vault

1. Go the following URL (http://localhost:8200/ui)
2. Choose to create a new storage

3. In this step, you will generate the keys needed to login/unseal the Vault, you will pick two numbers, the first is the total generated keys, and the second is the number of keys required to unseal the Vault (The default is 5 keys and threshold is 3)

4. Once initialized, you will be prompted to the page with your keys, download them and keep them safe, then continue to unseal

Example of downloaded keys

{
  "keys": [
    "fac35807810e2fe87f65734890609a30697eba08f1e4dc89fffa4a7f43bbf0d952",
    "69e0d3e2c63edc487418e8ca31ed9c0c0e4756820d711ae24cf08a706fd2f510b1",
    "1f06259c562bad3af4ef4ed272550a9620c4788008d35b4add4eb1bfe2cb2520c1",
    "19ae73c663b89bfc8e20891f8440988ff8b1c108b64421bfc76ec5694f9a26f469",
    "e620a7fae19261d7dcd8be0901462046f40e5138e114211267362b3fe7f3cdcc3b"
  ],
  "keys_base64": [
    "+sNYB4EOL+h/ZXNIkGCaMGl+ugjx5NyJ//pKf0O78NlS",
    "aeDT4sY+3Eh0GOjKMe2cDA5HVoINcRriTPCKcG/S9RCx",
    "HwYlnFYrrTr0707SclUKliDEeIAI01tK3U6xv+LLJSDB",
    "Ga5zxmO4m/yOIIkfhECYj/ixwQi2RCG/x27FaU+aJvRp",
    "5iCn+uGSYdfc2L4JAUYgRvQOUTjhFCESZzYrP+fzzcw7"
  ],
  "root_token": "hvs.9l2KJbQ41lw9Sd0BMl6RexCW"
}

5. Use any three keys in the unsealing, one by one, then use the root_token to login

6. And Voila! now you are in the dashboard of your Vault, and it’s ready to secure your valuable secrets.

Add your secrets

1. Create new secret engine with the (Key-Value) pair

2. Choose the KV

3. Create the secret engine for your app

4. Click on Create Secret to create secrets for your application

5. Now you can add your secrets, enable JSON if you want, then click save

6. After saving, you can move to the (Paths) to see how to access this JSON (Don’t forget to pass the token in the header)

Policies and Tokens

You shall not use the root token, so you will have to generate a new token with limited permissions (Here called Capabilities).

1. Create a new policy

2. Choose the name of the policy, then put the allowed actions you want (refer to How to write a policy) then click on Create policy (You will have to put the whole path as mentioned in the Paths earlier)

3. After that we need to create a token and assign it to a policy, but unfortunately we can not do this via the UI, so we will get into the docker container via this command

docker exec -it vault sh

4. In the container, login to Vault, you’ll be prompted to enter the token (enter the root token)

vault login

5. Then run the following command to create a token, assigned to a specific policy, you will see the token in the output, save it

vault token create -policy django

6. Now you can use your new generate token to access your secrets (Here’s an example with Postman)

Conclusion

As we have seen, keeping your secrets secret is not a big issue anymore, you only have to make a very few steps to make sure your secrets are stored safely in a place with least access to it.

You can refer to the docs to know more about the project.