BotC Tech role description and proposed transition to new admin
We are looking at someone or a team to take the tech head role. This process must be completed by the last day of March 2021 where current role will fully drop responsibilities and stop getting paid
BotC Tech could mostly decomposed in the 3 following areas:
- Software development/maintenance
- Sysadmin
- Crypto nodes management
Only after having experienced the tasks, learnt and documented as much as possible, I realized about the level of complexity in BotC project.
It's normally hard to find a person to cover both profiles reasonably, so it should be worth considering 2 roles as a team to cover the requirements. It's also advised against having only one fully responsible person. Two are the minimum recommended for better resilience.
Profile Requirements
Given the nature of any Fintech project, where an admin will have a high level of responsibility for keeping people's funds safe, it's important to find someone, or a team who can be fully trusted, and of course trust each other in order to share the responsibility and have more eyes on the software and infrastructure.
Ideally, you will want people who already have a reputation, assuming they will not want to burn it.
A candidate will ideally have the following experience, or ability to learn and catch up quickly with:
Software
BotC codebase is built on top of a deprecated verson of Symfony and is composed of one Symfony app for the frontend and one for the backend.
- 5 years of solid PHP knowledge
- 3 years of experience with Symfony or any similar web development framework
- Strong experience on troubleshooting, debugging and reverse engineering
- Experience with cryptos will help
Sysadmin
BotC infrastructure is mainly based on Libvirt virtualization plus some extra VPS servers on various providers.
- 5 years experience as sysadmin on production sites/software
- Solid experience with Debian/Ubuntu servers
- Virtualization
- Networking
- Security
- Bash and Python scripting
- OS release upgrades
- Backups maintenance, restore
- RAID 1 management
Crypto nodes management
Currently supported crypto nodes are Ethereum, Bitcoin and Faircoin.
- Experience as node operator will help: check balances, move funds between accounts, etc
- Setup and maintain remotely accessible RPC calls (for wallet API operation)
- Experience upgrading or migrating nodes software or servers if needed
Best practices
Collaborating with Holytransaction/Flypme, I've learnt some essential practices to keep everything as secure as possible:
- Sysadmin full paranoia mode on
- All laptops/workstations holding sensitive access, code or data must have full disk encryption
- When leaving secure locations, laptops must be turned off and not suspended to avoid certain physical attacks which could help gain critical infrastructure access.
- No third-party providers for sensitive information (ie, run your own Gitlab)
- One ssh key per server, passphrase protected
- Never leave a terminal unlocked+unattended if at unsafe locations (ie set autolock to a low value like 1min and force lock if you leave the keyboard)
Additional general recommendations
- Implement role isolation as much as possible: one dev, one sysadmin, one or many crypto admins, one per node. This will be useful only if a wrapper is included around each node, to allow only mandatory RPC calls for the wallet to operate, including logging and maybe notifications)
- No access for developers to production infrastructre (as part of role isolation)
- Keep encrypted backups off-site at controlled and undisclosed locations.
Proposed transition procedure
- Prepare a public call with the requirements.
- It is advised that candidates are validated at least by two reputable people from the Assembly.
- Create a transition telegram group where people from imMunitech will be invited to have the final say on the candidate or team.
- Candidate or team gets access to development infrastructure to learn and practice.
- Once candidates are approved and ready to take over, hand over credentials (see checklist below)
Handing-over credentials checklist:
-
One of the remaining root admins will remove credentials from all servers for leaving role. -
One of the remaining root admins will add credentials for the new roles. -
The new role will take care of re-creating the crypto nodes' private keys and place old ones on autoforwarder node, to forward funds arriving at any of the old (compromised) addresses. -
The new role will take care of changing all passwords to critical infrastructure or sensitive data must be changed.
From the time credentials are removed from leaving admin and private keys are replaced, leaving admin automatically drops responsibility for any issues happening, including unauthorized operations or loss of funds. Please, make sure to check the items on the list above as they are completed, to close down a potential attack vector.