Internal labs are a great resource for sales demos, training, testing upgrades and configuration changes, and replicating issues to help identify solutions and perform root-cause analysis. However, too often we take shortcuts and don’t use the same discipline and automation in internal labs that we follow in production environments. In the long term, this will lead to inefficiencies, quality control issues, and a missed opportunity to develop good habits that carry forward when working in our customer deployments.
In this blog post we’ll share suggested best practices that should be followed in your internal labs. Whenever possible, include key foundational processes and functions early on. This is time well invested that pays off in spades later!
Pick Infrastructure Provider
Quick sidebar: You probably already have a company standard for cloud or on-prem infrastructure, and this may limit your options as you select how to implement key functions. However, you should still evaluate alternate hosting options when building out a new lab.
If you opt for an on-prem deployment, don’t forget to account for the cost of managing your infrastructure. But, going to the cloud doesn’t negate all lab management costs. Be sure someone on your team has responsibility for overseeing and accounting for cloud infrastructure standards and costs. It’s all too easy for those cloud costs to quickly get out of control!
Automation, Configuration, and Security
Automation is a key capability to ensure a repeatable, efficient technology deployment. This is especially important in a lab environment where you need to rebuild it to clean up after testing or replicate different configurations. Use DevOps tools such as Terraform, Ansible, as well as specialized automation for your specific environment (for example, ArgoCD/Helm for Kubernetes).
Automation tools work best when used in conjunction with an Infrastructure as Code approach. Therefore you’ll also need to identify your Git repository early on as well. Keeping your configuration definitions in a repo and following good branching/merging discipline when making changes is critical. Having an audit trail of changes comes in handy when something breaks! In addition, this approach acts as a form of documenting your environment.
Security is important — figure it out and implement it early. Managing user identity, system accounts, passwords, and TLS certificates are often overlooked and not incorporated into labs, but these are all key in production deployments. Skipping good security practices is unfortunately a bad habit to get into. It often leads to extra work due to having to turn off certificate checks and introduce other workarounds, for example.
Deploy tools such as LDAP, Secrets Management, and an internal Certificate Authority from the beginning instead of trying to shoe-horn them in later. Also, get in the habit of changing default passwords and avoiding system accounts (such as root / Administrator) as much as possible.
Use Cloud Service or Implement Yourself?
Cloud providers have an extensive menu of infrastructure services that can be easily leveraged. In many cases, it makes sense to use them instead of implementing them yourself (especially for low-level services like DNS).
However, if your customer deployments generally include key services (for example, Harbor container registry) don’t take shortcuts. Instead of using cloud services, stand them up yourself so you have better experience doing so. This is especially important if you’re often working with on-prem / air-gapped deployments.
Implement Applications
At this point you’re finally ready to implement the application for which the lab is intended! Leverage the discipline and automation in internal labs for deployments and integrate production features (LDAP, SSL/TLS, etc.) for those application as much as possible.
Lab Discipline and Automation Pays Off!
Based on what you’ve learned along the way, this phase of your lab build-out should be easy mode! Best of all, you have a composable infrastructure that’s easy to rebuild as needed and track any changes made. And you are reinforcing the very best practices you should be following anyway in your production environments. Contact ServiceLaunch for more details in how we can help!