#sandbox-cloud-questions Slack channel to ask questions and get help.The Sandbox Cloud is an automated provisioning platform for creating an AWS account (today) or GCP project (near future) that you can use for demo/sandbox/testing purposes and is paid for by GitLab with consolidated invoice billing (no credit card required).
This platform is powered by HackyStack, an open source project created by Jeff Martin to automate the access request process using Okta integration, auto-assigning roles and permissions based on your department, and using the cloud provider API for provisioning your AWS account and/or GCP project.
You can learn more in the HackyStack High-Level Intro presentation:
The Sandbox Cloud is managed by the Business Technology Engineering team.
We are currently at
betastability. Please post in#sandbox-cloud-questionsif you see unexpected behavior.
Any team member can use the self service instructions below to provision an AWS account or GCP project for your individual use (sandbox, testing, etc.). You cannot invite other team members to individual accounts for security reasons.
#sandbox-cloud-questions to have it added.Provisioning to Active.You can sign-in with Okta, however please don’t create a Cloud Account unless you intend to provision AWS resources. You can see the screenshots of everything that a user sees.
Any team member can request a new AWS account or GCP project for a specific project or working group, or for general usage by their department group for shared non-production resources.
No RED data is allowed in these accounts/projects. Any RED data must be hosted in production AWS accounts or GCP projects managed by the appropriate Infrastructure Realm administrators (ex. eng-infra-saas, business-tech, etc.).
Self-service creation and IAM management is not available yet for end users in HackyStack (will be available in GitLab Access Manager in the future). In the meantime, we use access request style issue templates as our boring solution for security compliance reasons and the HackyStack administrators provision accounts and users using the Admin CLI.
The oversimplified user story is "I need to spin up VM(s) or cluster(s) in GCP or AWS to try something (anything, may not be GitLab product specific), what's the company infrastructure standards for doing that?"
The goal is to create a frictionless approach for technical team members that includes the tagging needed for cost allocation, best practice security configurations, and streamline the provisioning of infrastructure without needing to wait several days or weeks for an access request to be approved and provisioned.
This also reduces the burden on the accounting team that processes expense reports for team members each month. Each team member’s account is now part of consolidated billing.
Over the years, our non-production infrastructure resources have grown organically without accountability, cost controls, naming conventions, provisioning automation, or security best practices. This includes resources created in the GCP gitlab-internal project, AWS gitlab-np account, and DigitalOcean using dev-resources.
Epic 257 was created to iterate on our processes. In FY21-Q3, we created company-wide infrastructure standards which solved the “naming things is hard” problem with labels, tags, and naming conventions in our AWS and GCP organization accounts. The infrastructure standards define realms to create separate security boundary namespaces for different use cases. For our sandbox use cases, we’ve created a sandbox realm for individual users and department realms for shared collaboration projects, notably the Engineering Development realm which allows each of the department groups (functional teams) to have a shared AWS account or GCP project for creating infrastructure.
Jeff Martin developed the first release of HackyStack that powers the GitLab Sandbox Cloud. We developed the tooling in-house since the existing industry tools only solve 1-3 of the Technical Problems We're Solving and we wanted to automate the workflow end-to-end.
We are developing HackyStack as an open source project to allow other infrastructure or IT team to simplify their processes for provisioning sandbox accounts. HackyStack is not designed for individual use (yet). As we evolve, we'll be able to advocate HackyStack to partners and customers for deploying demo, testing, or training infrastructure without long manual provisioning documentation or burdening internal infrastructure team members.
This project was built using Laravel instead of other viable languages due to Jeff's prior experience and proficiency with Laravel to achieve the most efficient time to business value. This builds on the success of the GitLab Demo Systems that is powered by the demosys-portal.
For those who are not familiar with Laravel, it is the PHP equivalent of Ruby on Rails and Django and has seen tremendous community popularity in recent years since PHP has made revolutionary improvements in recent years with PHP 5.x and PHP 7.x. This project also allows us to dogfood GitLab CI/CD capabilities for PHP projects.
See the issue trackers for the latest up-to-date information.
dev-resources infrastructure.Phase 4 - Automated provisioning of AWS accounts and GCP projects for each user and team with streamlined/automated access requests (aka “Automate the manufacturing of everyone’s green LEGO board”). This is being achieved with the HackyStack open source project that Jeff is building.
Phase 4.5 - Migrate everyone’s resources in shared accounts into respective isolated accounts and apply labels/tags for cost management and reporting.
Phase 5 - Curate centralized library of Terraform modules, Ansible roles, Packer images, Docker images, and other scripts that have best practice security standards are used for deploying common infrastructure (aka “Provide everyone a box of LEGO bricks and the tools to deploy them”). Integrate GitLab Environment Toolkit for deploying GitLab in decentralized test environments (user sandboxes, community member environments, etc). This will be open source with the community so partners and customer POCs can take advantage of what we have. This will solve Sid’s request for ensuring we’re all on the same page and using the same library for the millions of GitLab users.
Phase 6 - Create “easy button” for deploying the library of infrastructure (aka the LEGO kits) into a topology builder.
Product and Revenue Enablement - Since a lot of provisioning functionality uses GitLab CI/CD and GitOps, we are dogfooding the GitLab product and allows users to manage their infrastructure-as-code in a GitLab repository and extend capabilities with other GitLab features as they see fit.
HackyStack and GitLab Premium Features - We will eventually add premium features to HackyStack that would require features included with a GitLab paid subscription.
Please post your ideas on Slack or in the issue tracker and so we can discuss the best ways to implement them.
Please direct any questions about the Sandbox Cloud or HackyStack to Jeff Martin or Dave Smith.