#sandbox-cloud-questions Slack channel to ask questions and get help.The Sandbox Cloud is an automated provisioning platform for creating an AWS account (today) or GCP project (near future) that you can use for demo/sandbox/testing purposes and is paid for by GitLab with consolidated invoice billing (no credit card required).
This platform is powered by HackyStack, an open source project created by Jeff Martin to automate the access request process using Okta integration, auto-assigning roles and permissions based on your department, and using the cloud provider API for provisioning your AWS account and/or GCP project.
You can learn more in the HackyStack High-Level Intro presentation.
The Sandbox Cloud is managed by the Business Technology Engineering team. Please tag Jeff Martin in Slack with any questions.
We are currently at
betastability. Please post in#sandbox-cloud-questionsif you see unexpected behavior.
Any team member can use the self service instructions below to provision an AWS account or GCP project for your individual use (sandbox, testing, etc.). You cannot invite other team members to individual accounts for security reasons.
#sandbox-cloud-questions to have it added.Provisioning to Active.You can sign-in with Okta, however please don’t create a Cloud Account unless you intend to provision AWS resources. You can see the screenshots of everything that a user sees.
Any team member can request a new AWS account or GCP project for a specific project or working group, or for general usage by their department group for shared non-production resources.
No RED data is allowed in these accounts/projects. Any RED data must be hosted in production AWS accounts or GCP projects managed by the appropriate Infrastructure Realm administrators (ex. eng-infra-saas, it-infra, etc.).
Self-service creation and IAM management is not available yet for end users in HackyStack (will be available through API integration with GitLab Access Manager in the future and tracked in hackystack#38). In the meantime, we use access request style issue templates as our boring solution for security compliance reasons and the HackyStack administrators provision accounts and users using the Admin CLI.
Cloud Account details page, click the View IAM Credentials button in the top right corner to open up a popup modal window.AWS Console URL, Username, and Password that you can use to sign in to your AWS account. The 12 digit number at the beginning of the URL is your AWS Account ID/Number.Open AWS Web Console button on the Cloud Account details page.AdministratorAccess to be able to perform any action inside of your AWS account. We do not provide team members access to the root user account since we only use this for break glass security incidents or related administrative activity by the Infrastructure Realm Owners.Cloud Account details page, click the View Credentials button in the top right corner to open up a popup modal window.GCP Console URL and username. Since GCP uses Google authentication, you simply need to be signed in with your GitLab email address on Google. HackyStack has added the Owner GCP IAM role to your email address when the project was created. Your project ID is in the format of {emailHandle}-{cloudAccountShortId}. You can choose this project from the dropdown list when accessing a different project in the GCP console.In the HackyStack v1.11 (November 2021) release, we introduced Terraform environment generation for GCP projects and a lot of underlying automatin for GitOps with alpha stability. AWS will be supported in a future iteration.
We ran into some tough engineering challenges and had to defer a few desired pieces to an upcoming patch release. In the spirit of iteration, we decided to ship what we do have working to avoid holding up the release with the few features that are harder than expected. There are manual configuration workarounds in the issues linked in the changelog that will take an extra 10-15 minutes for each environment that you create.
WAUBVZA3bneq4oWx)
Please see status updates on these workarounds in the
#sandbox-cloud-questionsSlack channel and issue descriptions. It is possible the handbook page may be out of date if we are able to resolve these quickly.
The oversimplified user story is "I need to spin up VM(s) or cluster(s) in GCP or AWS to try something (anything, may not be GitLab product specific), what's the company infrastructure standards for doing that?"
The goal is to create a frictionless approach for technical team members that includes the tagging needed for cost allocation, best practice security configurations, and streamline the provisioning of infrastructure without needing to wait several days or weeks for an access request to be approved and provisioned.
This also reduces the burden on the accounting team that processes expense reports for team members each month. Each team member’s account is now part of consolidated billing.
Over the years, our non-production infrastructure resources have grown organically without accountability, cost controls, naming conventions, provisioning automation, or security best practices. This includes resources created in the GCP gitlab-internal project, AWS gitlab-np account, and DigitalOcean using dev-resources.
Epic 257 was created to iterate on our processes. In FY21-Q3, we created company-wide infrastructure standards which solved the “naming things is hard” problem with labels, tags, and naming conventions in our AWS and GCP organization accounts. The infrastructure standards define realms to create separate security boundary namespaces for different use cases. For our sandbox use cases, we’ve created a sandbox realm for individual users and department realms for shared collaboration projects, notably the Engineering Development realm which allows each of the department groups (functional teams) to have a shared AWS account or GCP project for creating infrastructure.
Jeff Martin developed the first release of HackyStack that powers the GitLab Sandbox Cloud. We developed the tooling in-house since the existing industry tools only solve 1-3 of the Technical Problems We're Solving and we wanted to automate the workflow end-to-end.
We are developing HackyStack as an open source project to allow other infrastructure or IT team to simplify their processes for provisioning sandbox accounts. HackyStack is not designed for individual use (yet). As we evolve, we'll be able to advocate HackyStack to partners and customers for deploying demo, testing, or training infrastructure without long manual provisioning documentation or burdening internal infrastructure team members.
This project was built using Laravel instead of other viable languages due to Jeff's prior experience and proficiency with Laravel to achieve the most efficient time to business value. This builds on the success of the GitLab Demo Systems that is powered by the demosys-portal.
For those who are not familiar with Laravel, it is the PHP equivalent of Ruby on Rails and Django and has seen tremendous community popularity in recent years since PHP has made revolutionary improvements in recent years with PHP 5.x and PHP 7.x. This project also allows us to dogfood GitLab CI/CD capabilities for PHP projects.
See the issue trackers for the latest up-to-date information.
dev-resources and support-resources.Phase 4 - Automated provisioning of AWS accounts and GCP projects for each user and team with streamlined/automated access requests (aka “Automate the manufacturing of everyone’s green LEGO board”). This is being achieved with the HackyStack open source project that Jeff is building.
Phase 4.5 - Migrate everyone’s resources in shared accounts into respective isolated accounts and apply labels/tags for cost management and reporting. See it-infra#86 Project Playground for details.
Phase 5 - Curate centralized library of Terraform modules, Ansible roles, Packer images, Docker images, and other scripts that have best practice security standards are used for deploying common infrastructure (aka “Provide everyone a box of LEGO bricks and the tools to deploy them”). Integrate GitLab Environment Toolkit for deploying GitLab in decentralized test environments (user sandboxes, community member environments, etc). This will be open source with the community so partners and customer POCs can take advantage of what we have. This will solve Sid’s request for ensuring we’re all on the same page and using the same library for the millions of GitLab users.
Phase 6 - Create “easy button” for deploying the library of infrastructure (aka the LEGO kits) into a topology builder.
Product and Revenue Enablement - Since a lot of provisioning functionality uses GitLab CI/CD and GitOps, we are dogfooding the GitLab product and allows users to manage their infrastructure-as-code in a GitLab repository and extend capabilities with other GitLab features as they see fit.
HackyStack and GitLab Premium Features - We will eventually add premium features to HackyStack that would require features included with a GitLab paid subscription.
Please post your ideas on Slack or in the issue tracker and so we can discuss the best ways to implement them.
The GitLab Sandbox Cloud is a part-time side project for Jeff Martin, so we are not able to commit to firm timelines for feature development. We will try to resolve bugs promptly, however please comment in an issue and tag @jeffersonmartin if there is a feature or capability that needs to be prioritized.
Please direct any questions about the Sandbox Cloud or HackyStack to Jeff Martin or Dave Smith.