GitLab 12.10 now helps teams streamline and improve compliance with requirements management, reduce cycle time and accelerate software delivery with CI with auto-scaling on AWS Fargate, and more efficiently manage a portfolio of projects with issue and epic health status.
Compliance is easier
Compliance is a common challenge in most large organizations, where teams and projects need to demonstrate they followed the organization’s processes and procedures, and delivered what was actually "required". Did the project actually address the business requirements is a common question, and with 12.10, we’re starting to deliver requirements management as a distinct category in GitLab to help teams define, track, and manage business requirements. Also, demonstrating project and release compliance just got a little easier in GitLab 12.10, as there's no longer a need to use scripts to compare release evidence over time, helping teams document and prove that the project is in "compliance".
The new project compliance framework label makes it easy for organizations to indicate that a specific project is required to comply with specific compliance frameworks.
Speaking of compliance frameworks, to help projects that are subject to HIPAA audits and compliance, the new HIPAA audit protocol project template gives them a head start.
It's also easier to protect your secrets with improved HashiCorp Vault Integration, which helps keep your projects compliant with your security policies.
Reduce Cycle Time and Accelerate Delivery on AWS
The last thing you need is another bottleneck that potentially slows down delivery and that’s why we’ve supported autoscaling GitLab CI runners for a very, very long time. In GitLab 12.10, we’re extending our autoscaling ability on AWS Fargate to auto-scale runners so pipelines can efficiently scale to meet demand. Speaking of AWS, it's now faster and easier to configure your application to deploy to AWS with predefined AWS deployment variables, where GitLab has added AWS deployment variables and also helps with format validation.
Efficiently manage projects
Managing multiple projects and associated issues can be hard to juggle. With all the information there is to track, it's hard to know where there might be problems. Now in GitLab 12.10, it’s easy for teams to track and share the health status of issues so that it’s simple to visualize the overall health of the epic. Additionally, we’re making it easier to import issues from Jira into GitLab so that teams can spend less time switching between tools and more time focused on building great software.
Dmitry has made a major contribution adding the Puma web server to the GitLab unicorn Helm chart (soon to be the webservice chart). This work provides users of the GitLab Helm chart with the option to use Puma instead of Unicorn. In testing, we have observed a 40% reduction in memory usage when using Puma as the web server. Dmitry, we appreciate your awesome collaboration and enormous contribution to add Puma to our cloud native installation.
Check out the epic, and get started today! Thank you, Dmitry, for your contribution and hard work.
The first step towards managing requirements from within GitLab is
here! This initial release allows users to create and view
requirements at a project level.
We often hear about the struggles associated with external requirement
management tools - difficult integrations, multiple toolchains, and
challenging workflows. We believe in the power of a single application, by
offering an opportunity to keep all requirements, designs, code, and
tests in a single environment. As Requirements Management evolves in GitLab, stay tuned for support for
traceability between all
artifacts, creating a seamless workflow to visually demonstrate completeness and compliance.
In this release, GitLab adds support for lightweight JSON Web Token (JWT) authentication to integrate with your existing HashiCorp Vault. Now, you can seamlessly provide secrets to CI/CD jobs by taking advantage of HashiCorp’s JWT authentication method rather than manually having to provide secrets as a variable in GitLab.
Managing multiple epics across multiple groups and projects is difficult. To help users track projects and in-flight work GitLab
now enables you to report on and quickly respond to the health of individual issues and epics by showing a red, amber, or green
health status on your Epic Tree. Assign an issue a health status of On track (green), Needs attention (amber), or
At risk (red) and see an aggregate report of health at the Epic level. Quickly view and analyze where a collection of work is
at risk so you can open up the right discussions at the right time and keep work on track!
Until now, the only way to get Jira issues into GitLab was manually,
with our CSV importer, or by hand-rolling your own migration utility.
GitLab 12.10 includes an
to automatically import your Jira issues into GitLab. This is the first
of many planned
make transitioning from Jira to GitLab as frictionless as possible.
To get started, set up the Jira integration on your GitLab project, click
the import icon at the top of your project’s Issue List, and select
Import From Jira.
GitLab Runner autoscaling responds to CI job demand by provisioning new cloud-hosted virtual machines. However, while this model of automatically spinning virtual machine instances up and down continues to be sufficient for specific use cases, customers also want to take advantage of the capabilities of cloud container orchestration solutions for executing GitLab CI/CD jobs.
You can now auto-scale GitLab CI on AWS Fargate with the MVC release of GitLab’s AWS Fargate Driver. With this new autoscaling pattern, GitLab’s AWS Fargate driver automatically runs each build in a separate and isolated container on Amazon’s Elastic Container Service (ECS) using a user-defined container image.
When deploying to AWS, applying the necessary environment variables should be as convenient as possible. You can now select the predefined variables for ‘AWS_ACCESS_KEY_ID’, ‘AWS_SECRET_ACCESS_KEY’ and ‘AWS_DEFAULT_REGION’ from the environment variable key list. You’ll also see the variables you enter validated to ensure they are entered in a valid format.
Release and build managers are often in charge of wrangling several activities outside of GitLab in order to effectively deliver a release. GitLab is working to make the Release page a single source of truth for everything regarding your releases. We’ve added the ability to link runbooks to the assets of a Release so you can now track all of these related activities easily and see how far along they are.
GitLab Secure scanners need internet connectivity to download updates
and the latest signatures. GitLab 12.10 makes it substantially easier
to use these scanners when running self-hosted GitLab instances offline
or with limited connectivity. Several adjustments to the underlying
scanner job definitions support this workflow.
New documentation for offline environments
describes the high-level workflow needed for Secure scanning in an
offline environment. We have also added scanner-specific instructions
on each scanner’s documentation page.
We will continue to add support for offline Secure scans in future
releases, by offering support for additional languages, tools, and use
When your service is down or degraded, your top priority is to fix it. At the same time, you must update customers and business stakeholders about the progress of your fixes. Keeping them in the dark can lead to a flood of emails from unhappy people. Users rely on status pages to confirm if providers know about problems and to learn what to do. When there’s an active incident, knowing what steps are being taken inspires confidence. It helps people to make choices about what they will do in response to the incident.
Today, the new GitLab Status Page is available. Keep users, customers, and stakeholders informed during incidents. Push information from private incidents, like issue descriptions and select comments, from a private incident issue to a public web page. Work directly from the incident issue you are already using for triage and firefighting, instead of bouncing between a lot of different tools.
To begin with, we are targeting one narrow use case. We designed Status Page to publish information from issues in a dedicated incident management project on a private GitLab instance out to public status detail pages. Visit the Status Page Direction to see our plans to add capabilities and support more use cases.
Python developers need a mechanism to create, share, and consume packages that contain compiled code and other content in projects that use these packages. PyPI, an open source project maintained by the Python Packaging Authority, is the standard for how to define, create, host, and consume Python packages.
In GitLab 12.10, we are proud to offer PyPI repositories built directly into GitLab! Developers now have an easier way to publish their projects’ Python packages. By integrating with PyPI, GitLab will provide a centralized location to store and view those packages in the same place as their source code and pipelines.
In March, we announced that the GitLab PyPI Repository and support for other package manager formats will be moved to open source. You can follow along as we work to make these features more broadly available in the epic.
We’re pleased to announce Container Network Policy Statistics Reporting! You can now see
data on both total and blocked traffic, allowing you to more easily determine how to
configure, tune, and evaluate your Network Policies.
Container Network Policy statistics will appear on a new Threat Monitoring page under the
Security & Compliance menu item. By default, this data covers a 30-day period.
When an administrator or group owner assesses compliance of their GitLab projects, they need to know the status of pipelines for the code being deployed. Pipelines can contain stages or jobs that determine compliance with organizational policy. Until now, administrators or group owners had to investigate every project to validate each pipeline.
The Compliance Dashboard now includes the most recent pipeline status for all projects in a group. Administrators and group owners can now identify potential compliance issues with approved and merged MRs at a glance.
In the General section of your group’s settings, you can specify a lifetime duration for personal access tokens created by members of a group-managed account. This policy will only apply to users of that group.
With GitLab, you can automate highly repeatable HIPAA Audit Protocol workflows. GitLab can help simplify these workflows natively by leveraging issues and project templates. This process can help map new issues to each requirement in the HIPAA audit protocol. It can also help your organization manage audit evidence collection and overall status within your GitLab workflow.
GitLab now supports the HIPAA audit protocol, through the new enterprise compliance template. To aid in HIPAA compliance, GitLab can help you create new projects, each with the 180 issues that map to the HIPAA audit protocol. Each issue serves as an audit trail for each HIPAA protocol and can help teams stay connected as they manage their HIPAA compliance programs within GitLab.
Issues are one of the primary tools for collaboration within GitLab. The
default order of oldest-to-newest discussions and systems notes is
incredibly helpful for some use cases, such as understanding the history
of a given issue. However, it is much less helpful when teams are in
triage and firefighting mode, as they have to scroll all of the way to the
end of the issue to see the latest updates.
Now you can reverse the default order and interact with the activity
feed with the most recent items at the top. Your preferences for Issues and MRs are saved separately via local storage and are automatically applied to every Issue and MR that you view.
Designs uploaded to Issues can be quite large in file size. Loading these files can take a long time, especially if you have more than one Design in an Issue. With this release, we now automatically generate Design thumbnails and use them to speed up the load time of the Design tab. This will also enable us to improve loading times of Designs in other parts of GitLab as we continue to build out the Design Management feature.
We tested the GitLab homepage mockup which is 1222px by 5113px and was 2.6MB. With thumbnail generation, this image comes down to 0.018MB, which is a ~99.9% reduction in size!
Hosting static websites with GitLab Pages is an easy and quick way to get your site up and running. Editing the content that populates your static site, however, often requires setting up complex local development environments, an understanding of the underlying project architecture, and familiarity with Markdown syntax. For some, this is a barrier to collaboration.
We’re taking our first step toward delivering a first-class editing experience for static site content that facilitates collaboration on content without requiring any prior knowledge of programming languages or even Git. GitLab now includes a new project template that creates a static website, initially supporting Middleman, pre-configured to be hosted on GitLab Pages and with content that can be edited in a new, streamlined Static Site Editor. After deploying the site, just click the “Edit this page” link visible on every page and this will launch our new editor which removes the extraneous interface and focuses entirely on the content of the page. When you’re done, one click generates a new branch and a merge request for your changes.
Up to GitLab 12.9, when you contribute to the wiki content, it does not influence your activity in GitLab. With this release, you’ll see all wiki contributions on the project, group, and user activity pages. Now you can track wiki changes and wiki editors will get credit for their contributions!
It is now possible to set up a dependency relationship on a project you depend on, such that when the default branch on that project builds successfully, a pipeline on your default branch also triggers. This relationship is set up as a subscription from the project with the dependency, making it easy to set these up even in situations where the project you depend on is managed by another group.
The GitLab Container Registry used to show an illustration that contained easy to copy build and push commands with the correct registry URL for your project. However, once an image was added to the registry, the commands would disappear.
In 12.10, we now show these Docker commands even when the registry is not empty. This will make the onboarding process of a new user easier and will boost their familiarity with Docker commands.
Deploy Tokens allow you to access your group and project’s repositories and container registries. However, the defined scopes of read_repository and read_registry have not allowed you to grant push access to the GitLab Container Registry. As a result, DevOps teams have used insecure or expensive user based workarounds.
In GitLab 12.10, we are excited to announce more granular permissions for GitLab Deploy Tokens. You can now set read or write access for the Container Registry. You can create and manage Deploy Tokens from within the GitLab application or by using the API.
When using the GitLab Container Registry, it is important to have a cross-project view of all your Docker images. Until recently, the user interface has only been available at the project level. This inefficient workflow has resulted in a lack of collaboration and sharing of Docker images amongst teams.
In 12.10, we are excited to announce that you can now view all of your group’s images within the GitLab application. Now you can share, discover and manage all of your images in one place.
All Dependency Scanning analyzers now support reporting severity levels, making findings more easily assessed for risk and priority. Previously, not all languages were supported by findings with severities leaving some severities set to Unknown, making it difficult to prioritize their remediation.
When merge trains are enabled, the /merge quick action now automatically adds the merge request to the merge train. In previous releases, using this quick action skipped the merge train altogether and merged requests immediately; this behavior was both unexpected and confusing.
A critical part of monitoring and observing system and application performance is the alert you receive that tells you when something has gone wrong. Without alerting, there is no way to close the DevOps loop and efficiently respond to services where critical thresholds have been breached. These capabilities are needed by every development team and we want our Alert Management solution to be widely available to everyone that uses GitLab.
As part of our 2020 gift, we’ve moved alerting in the Monitor stage from GitLab Ultimate to GitLab Core. Beginning with GitLab 12.10, all users can create IT alerts from charts on the metrics dashboard, and receive alerts in GitLab via the generic REST endpoint.
When resolving incidents, charts embedded directly into an issue can save you time by displaying all of your information in one place instead of forcing you to bounce around between multiple locations. However, charts can become frustrating if you only want to read the text and quickly consume the information. You can now collapse and expand charts, and choose between seeing the charts or removing them from the view.
Auto Monitoring is a feature of Auto DevOps, a predefined CI/CD configuration allowing you to automatically detect, build, test, deploy, and monitor your applications.
Auto Monitoring enables you to monitor your application’s server and response metrics right out of the box after deploying your application. Auto Monitoring uses Prometheus to display system metrics such as CPU and memory usage directly from Kubernetes, and response metrics such as HTTP error rates, latency, and throughput from the NGINX Server. While this sounds like a lot to configure, we’ve reduced the configuration required for you to see the metrics and alerting in action by adding out-of-the-box alerts for NGINX Throughput and HTTP error rate metrics that work right away after you set up Auto Monitoring for Auto DevOps.
Systems administrators need to know the overall status of their Geo
installation. This is especially important if replication errors are
detected. In GitLab 12.10 we’ve added several iterative improvements to the Geo administrator interface to make it easier for systems administrators to diagnose issues with Geo and to help them understand what corrective actions they need to take, for example by identifying
repositories that failed to replicate.
One of the biggest changes is the addition of a popover that displays a detailed
breakdown of all synchronized, queued and failed items. Where available,
there is a link to a details page, which allows systems administrators
to investigate individual items.
Additionally, we’ve made a few other user experience improvements to the administrator interface:
The Helm Chart documentation has been updated to include command line options and documentation links specific to Helm 3, and to highlight some of the differences between Helm 2 and Helm 3. The specific changes can be viewed in the issue.
In GitLab 13.0, the next release of GitLab, we will be removing the ConfigMap entries for puma.rb and unicorn.rb files in favor of environment variables. Please note that if you have modified the unicorn.rb from the ConfigMap (via Helm + Kustomize) in the past, you will be impacted by this change. We are doing this to eliminate maintaining two copies of these files. Until now, the puma.rb and unicorn.rb files were static within the gitlab-webservice container and overwritten by ConfigMap items from the gitlab/unicorn chart. For details on the new environment variables, refer to the associated merge request.
Now that you can choose between Unicorn or Puma for your web application server, the name of the Unicorn chart will be changing from unicorn to webservice. This change will take place in GitLab 13.0. As a reminder, Puma will become the default application server in GitLab 13.0 due to the performance improvements we have seen with its multi-threading capabilities. To try out Puma in your Helm install, see the instructions in the Charts documentation.
The GPG signing keys for the GitLab PackageCloud repository have expired and have been replaced with new ones. This means existing users who had already configured GitLab package repositories on their machines to be used via apt, yum or zypper, will have to fetch and add the new keys to continue installing or updating packages from the GitLab package repository. For detailed instructions, see the Package Signatures documentation. Further details are also available in Balu’s blog post.
GitLab is switching application servers from Unicorn to Puma, reducing the memory footprint of GitLab by 40%. This improved efficiency can allow GitLab administrators to leverage smaller memory instances, reducing the operational costs of the service. These savings are achieved due to leveraging the multi-threading support in Puma, compared to the single-threaded model of Unicorn.
Sidekiq Cluster allows starting additional Sidekiq processes to
run background jobs, as well as offering convenient options for
selecting a particular set of job queues to be processed. These options allow for improved management of Sidekiq queues, and continued scaling for large instances.
Previously this feature was only available in the Starter tier and higher. As of GitLab 12.10, it is now available in Core. It will be enabled by default starting in GitLab 13.0.
The Users statistics page provides administrators with an overview of user accounts by role, also totals of all users, active users, and blocked users. GitLab billing is based on the number of active users. The information on this page helps you manage seat allocation and subscription costs.
We are continuing to make great strides improving the performance of GitLab in every release. We’re committed to not only making individual instances of GitLab even faster, but also to greatly improving the performance of GitLab.com, an instance that has over 1 million users!
In GitLab 12.10 we are shipping performance improvements for issues, projects, milestones, and a lot more! Some of the improvements in GitLab 12.10 are:
Organizations using GitLab have company policies and industry regulations that dictate how they operate. A top-of-mind goal for many of our customers is ensuring their GitLab projects are adhering to internal company policies, which are influenced by industry regulations. Previously, there was no easy way to identify a project as one that has certain compliance requirements or additional oversight, which is a fundamental need to tracking compliance status.
Now you can label projects with specific compliance frameworks by navigating to a project’s Settings area and, in the General section, choosing from the Compliance framework dropdown menu. This feature lays a foundation to ease project compliance management in the future.
Development leaders and GitLab administrators often wish to understand how their teams are using GitLab. In this MVC you’ll see three counts on the group landing page: Merge Requests, Issues, and Users added to the group, all over the past 90 days. This feature is being released in beta.
Compliance-minded organizations need a way to control SSH credential access to their GitLab environment. SSH keys are normally configured without expiration dates. This is problematic for organizations with access management and/or credential management policies, which require an expiration date on all access credentials. With this release, GitLab supports expiration dates for SSH keys that users can set within the GitLab UI.
When fetching changes from a Git repository, the server advertises a list of all the branches and tags in the repository, known as refs. In some instances, we have observed up to 75% of all requests to the GitLab web server are requests for the refs. In the best case, when all the refs are packed, this is an inexpensive operation. However, when there are unpacked refs, Git must iterate over the unpacked refs. This causes additional disk I/O, which is slow when using high latency storage like NFS.
In GitLab 12.10, info/refs are cached to improve the performance of ref advertisement and decrease the pressure on Gitaly in situations where refs are fetched very frequently. In testing this feature on GitLab.com, we observed read operations outnumber write operations 10 to 1, and saw median latency decrease by 70%. For GitLab instances using NFS for Git storage, we expect even greater improvements.
Access to Git repositories is critical to developers and businesses. If an outage occurs, developers can’t push code, and deployments will be blocked. To prevent outages, GitLab can be run in a highly available (HA) configuration. The recommended approach currently uses Network File System (NFS), but this adds significant latency to every read and write operation, severely impacting the performance of GitLab.
In GitLab 12.10, Gitaly now offers beta support for high availability without using NFS. While data loss is not likely, it is not recommended for use in production environments yet. In this release, the replication queue and leader state have been moved to a PostgreSQL database. Previously, the replication queue and leader state was stored in memory in the Praefect proxy/router. This prevented configurations using multiple Praefect nodes, and could result in data loss if Praefect was restarted before the replication queue was drained.
The first version of High Availability for Gitaly is eventually consistent. It is implemented as an asynchronous replication queue, and favors availability over consistency. If an outage occurs on a primary Gitaly node before the replication queue has been drained, data loss is an expected side effect of automatic failover. Work is already underway on strong consistency to prevent such data loss scenarios.
When viewing files in your repository, GitLab now displays an icon based
on the file extension.
Different file types use icons of different colors and shapes to help people
quickly recognize files in the editor, and when browsing their computer.
This change also makes icon styling consistent between the repository view,
the Web IDE, and the merge request file tree. Enjoy!
With the new Copy & Paste functionality, you can now paste one image from the top of your clipboard history directly into the Designs tab as a .png file. This functionality is particularly useful to quickly share screenshots from the clipboard in any issue.
We’re also releasing GitLab Runner 12.10 today! GitLab Runner is the lightweight, highly scalable, cross-platform software
agent that runs your build jobs and sends the results back to a GitLab instance.
GitLab Runner works in conjunction with GitLab CI/CD, the open-source continuous integration service included with GitLab.
GitLab’s Package Registry enables you to store a myriad of package types in a single, universal, registry. Until recently, the only way to explore your list of packages was to change the sort order. This made it difficult to find a specific package efficiently, especially if you stored many packages within your registry.
We are excited to announce that as of GitLab 12.10, you can now filter by name to find your packages quickly.
The GitLab Dependency Proxy allows you to proxy and cache images hosted on DockerHub so they are readily available for use within GitLab CI/CD. However, up until this release, there wasn’t a way for you to purge the cache and that resulted in additional storage costs.
This is no longer the case. Now, you can use the GitLab API for purging the cache of your group’s Dependency Proxy and lower your total cost of storage.
As a black-box tool, a DAST scan doesn’t know anything about the underlying site or application architecture. This can lead to false positives that the user immediately knows aren’t exploitable vulnerabilities. An example of this is the DAST scan reporting a possible SQL injection vulnerability when there’s no SQL database in the application architecture.
Because of this problem, GitLab 12.10 supports toggling specific rules on and off using the DAST_EXCLUDE_RULES variable. This allows using a comma-separated list of Vulnerability Rule IDs to be excluded from scans. When using this variable to exclude specific rules, it’s possible to better tailor a scan to the targeted app to get fewer false positives.
In 12.6, GitLab introduced a streamlined approach to collect all the necessary information to support compliance and audit efforts within the Release object. With this new feature, an evidence collection timestamp will appear alongside a link to download the evidence hash. This enables auditors to easily compare Release Evidence over time, rather than requiring a script to collect and compare each piece of evidence.
As part of our AWS Cloud Deploy Docker image, we’ve bundled a script you can leverage in your pipeline to streamline task creation for ECS deployments. This helps you move boilerplate code in deployment jobs and simplifies your CI/CD configuration.
This feature enables users to easily administer their environments from the GitLab UI rather than API. Expanding the management of environments to the UI saves time and supports users spending their day in the GitLab frontend.
While triaging an incident, charts help you visualize what went wrong, which can speed up investigation and resolution. In 12.9, GitLab started automatically embedding relevant charts in all incident issues that were created from GitLab-configured Prometheus alerts. We have now extended this feature to work in issues generated from all Prometheus alerts GitLab receives, whether the alerts were configured in GitLab or configured externally.
Understanding system performance often starts with monitoring the metrics that convey the health and performance of the components in question. These capabilities are needed by every development team and we want our metrics solution to be widely available to everyone that uses GitLab.
As part of our 2020 gift, we’ve moved custom metrics in the Monitor stage from GitLab Ultimate to GitLab Core. Beginning with GitLab 12.10, all users can add and display any metric they collect using Prometheus in the GitLab UI.
Logs, while ubiquitous, are only useful if the relevant logs can be found easily. In GitLab 12.10, we introduced filtered search for pod logs, enabling you to search and define filters for pod logs with a single, scalable search bar. This replaces the cumbersome combination of terminal view, filters, and a search bar, ultimately enabling you to find the relevant logs more easily.
Sometimes using predefined time intervals makes pinpointing a specific time period difficult. GitLab monitoring dashboards now support custom intervals, which enable you to visualize your aggregated metric data in an interval of your choosing, on top of being able to visualize with the default intervals.
GitLab supports managing large binaries in projects via Git
such as audio, video, or graphics files. These files can be removed
from LFS by rewriting Git history; however, unreferenced files
will still use up storage. Until now, the only way to remove these
unreferenced LFS objects was to delete the entire project, which is not
an option in many situations. Consequently, users may run up against
storage limits, and realize that they are using a lot of storage on LFS
objects they no longer want or need, without a clear path forward.
Fitting for the Spring season, we added two Rake tasks,
gitlab:cleanup:orphan_lfs_file_references and gitlab:cleanup:orphan_lfs_files,
that allow removing LFS files from individual projects.
The Rake tasks can be run on a project-by-project basis and scheduled
as needed. Happy Spring cleaning!
Geo supports selective
of projects, which allows systems administrators to choose a subset of
data that is replicated to a secondary Geo node. Until now, users trying
to access repositories on a secondary node that were not synchronized
would receive an error that the project was not available. This could be
because of selective synchronization or because of replication lag. This was
confusing for users and disturbed their Git workflow.
In GitLab 12.10, any Git requests made via HTTP(S) to an unsynchronized secondary Geo node will be forwarded to the primary node so that users can still access the
repository. This means that users won’t need to know what is or isn’t
replicated - Geo will try to fulfill the request. Support for proxying SSH Git operations will be available in GitLab 13.0.
Global Search in GitLab has long been able to return code results for searches performed. However, it wasn’t clear to users where in the returned results their search query was actually matched. This meant that users had to hunt for this in the results and may have missed important results where the string may have been returned multiple times.
Now, each line that matched the search will be highlighted to provide a better indication of where in the results the search term was matched. This also matches multiple occurrences in the same file to better highlight each line that may be valuable.
Huge thanks to @terales for contributing this feature!
The default version of GitLab-provided PostgreSQL is now PostgreSQL 11. For new installs, and upgrades of existing installations that do not use HA or Geo, PostgreSQL 11 will automatically be used by default. For Geo and HA installs, please see the 12.10 upgrade notes. In GitLab performance testing using the 10K user reference architecture, we found that PostgreSQL 11 was able to handle 7% more Requests Per Second compared to PostgreSQL 10, and showed 20% less CPU usage for the merge request discussions endpoint. We did not observe performance regressions when moving from PostgreSQL 10 to 11. For a more detailed performance analysis, see the performance testing issue.
Note that PostgreSQL 9.6 and PostgreSQL 10 will no longer be supported as of GitLab 13.0. You will need to upgrade your PostgreSQL version prior to upgrading to 13.0.
If you are using Geo, please keep in mind that major PostgreSQL updates cannot be performed without downtime on the Geo secondary due to the need to resynchronize the database on the secondary cluster. Specific steps for Geo are available in the Geo upgrade docs. If you are using the Helm install, you will need to manually switch to PostgreSQL for 12.10.
Previously, scaling Advanced Global Search in GitLab to very large instances has been challenging due to the size of the Elasticsearch index required. This index was made up of search data and configurations, which were only partly utilized.
We’ve re-evaluated our usage of which content needs to be indexed, and updated the index_options for our Advanced Global Search configuration. On GitLab.com we’ve seen an almost 50% reduction for our production Elasticsearch Index. This change should make it easier to get started with Advanced Global Search and help you save on operational overhead when running Advanced Global Search in GitLab.
GitLab 12.10 has switched from utilizing schema.rb to structure.sql for managing the database schema. Switching to plain SQL in structure.sql allows GitLab to make use of PostgreSQL-specific commands, like partitioning.
Contributors and administrators may want to be aware of the change, in the event they need to work with migrations. The change to structure.sql will be automatically performed, and no action is required.
The consul[user] and repmgr[user] settings are deprecated and will be removed in GitLab 13.0. Other services in GitLab use username rather than user. For consistency, the user name for Consul and repmgr will be provided using a consul[username] and repmgr[username] setting from 13.0 onwards.
May 22, 2020
Auto DevOps Auto Deploy setting value deprecated
Because several APIs were removed in Kubernetes 1.16, the default value of extensions/v1beta1 for the deploymentApiVersion setting in Auto DevOps’ Auto-Deploy chart is now deprecated.
The deploymentApiVersion setting is scheduled to be changed to a new default of apps/v1 in GitLab 13.0. If you are using Kubernetes 1.9 and below, you will need to upgrade your Kubernetes cluster in order to get apps/v1 version support. For Auto DevOps GitLab requires Kubernetes 1.12+.
May. 22, 2020
Auto DevOps and Secure Configuration templates are changing to `rules` instead of `only/except`
The use of only and except is discouraged in favor of rules. The rules parameter provides more verbose and expressive job execution logic that is less complex to evaluate and understand.
Any customization to these templates using only and except should be transitioned to the rules syntax. There are occasional cases where the existing only and except logic cannot be matched with rules. We would love to hear more about these cases on this rules improvement issue.
As part of updating Auto DevOps to support Kubernetes 1.16, we added the opt-in ability for
Auto DevOps to use the PostgreSQL chart version 8.2.1 in GitLab 12.9. The default PostgreSQL
chart version is currently 0.7.1. In GitLab 13.0, we plan to switch the default PostgreSQL
chart version from 0.7.1 to 8.2.1. To remain on the old default, you will need to explicitly
set the AUTO_DEVOPS_POSTGRES_CHANNEL CI variable to 1.
To migrate your existing 0.7.1 PostgreSQL database to the newer 8.2.1-based
database, follow the upgrade guide
in order to backup your database, install the new version of PostgreSQL, and restore your database.
May 22, 2020
In GitLab Runner 13.0 we will remove the backported os.Expand() from Go v1.10.8. This backport was needed to include a change in behavior introduced in Go v1.11. You can find more details in issue #4915.
May 22, 2020
GitLab Pages disk source no longer supported in 14.0
GitLab Pages new API-based configuration will be available in 13.0 and will replace the disk source configuration. You can still use the old configuration through next year, although we recommend switching to the API-based configuration sooner rather than later, as new features will not be supported with the old configuration. In many cases, GitLab Pages will automatically switch to the API-based configuration without any action. There is nothing you need to do right now. Starting in 13.0, GitLab Pages will automatically use the new configuration source and fallback to the old one in case of errors. There will also be a way to enforce the old configuration source, but it will be removed in 14.0.
May 22, 2020
GitLab Snippets content search removal in 13.0
As we continue to work towards version control for Snippets we’ll be making a change to search for Snippets in the UI and API that removes snippet content from search results. Title and Description content will be accessible via search and API.
May 22, 2020.
Introducing a new 'id' field which will eventually replace 'cve' in the JSON common security report
GitLab’s current JSON common report format for security reports needs
improvements, particularly as we encourage and add third-party security
vendor integrations. The primary field cve property is confusing, as it
does not contain CVE data, and should therefore be removed. We are introducing
the id field, which is automatically calculated for GitLab scanners and
is required for third-party partner scanners. The id field will eventually
replace cve as a unique finding identifier. Anyone leveraging the cve property in
security reports, with custom scripts or as an integrator into our Secure
features, will eventually need to stop using the cve property and instead should
start using the new id property. Please be aware that today id and cve are both required fields.
Offset pagination limit of 50,000 for /projects endpoint
An offset-based pagination limit of 50,000 will be applied on
GitLab.com, and by default on self-managed instances, to the /projects
API endpoint in GitLab 13.0. Integrations which make API calls with
offsets above 50,000 will need to switch to a keyset-based pagination
will offer significantly improved response times and reduced load on the
GitLab server. Self-managed instances will be able to customize the limit
to a desired value.
To provide the optimized performance, keyset-based pagination only offers
ordering based on project id. Use cases which require more flexible
ordering options can continue to use offset-based pagination, provided the
offsets remain below the limit. If use cases require flexible ordering
options with deep offsets, we recommend sorting client-side.
May 22, 2020
Planned removal of 'marked_for_deletion_at' attribute in Projects API in GitLab 13.0
For customers using GitLab Silver, Premium or above, GitLab’s API response when listing projects currently returns an attribute named marked_for_deletion_at, which denotes the date on which a project was been marked for soft deletion.
To standardize on terminology across our APIs, this attribute will be deprecated in GitLab 13.0. A new attribute named marked_for_deletion_on with the same information has already been added.
May 22, 2020
Planned removal of 'projects' and 'shared_projects' attributes when requesting group details via API in GitLab 13.0
To improve performance, we limited the number of projects returned from the `groups/:id endpoint. Starting with GitLab 12.9, we limited the number of projects returned on the group details API to 100.
To further improve performance of this endpoint, we are removing the ‘projects’ and ‘shared projects’ attributes from the response when requesting details of a group via API, starting in GitLab 13.0. Users can still find this information in the same Group API, in the list a group’s projects endpoint.
May. 22, 2020
Planned removal of `token` attribute in Runner's details API
In GitLab 13.0 (May 2020), we are removing the token attribute from the Runners API endpoint that gets details of a runner by its ID. You can provide feedback in the related issue or your usual support channels.
As part of the GitLab 13.0 upgrade, users who have customized Unicorn settings will need to manually migrate these settings to Puma. It will also be possible to remain on Unicorn, by disabling Puma and re-enabling Unicorn until Unicorn support is removed in a future release.
sidekiq-cluster will become the default method of invoking Sidekiq in GitLab 13.0
In GitLab 13.0, we will be deprecating the sidekiq-cluster parameter
that is used to start multiple Sidekiq processes. In GitLab 13.0, instead
of manually enabling sidekiq-cluster and specifying parameters
as described in Starting multiple processes,
Sidekiq Cluster will be enabled by default, and a default set of
parameters will be provided. If you already have Sidekiq Cluster
enabled, there should be no visible change.
The GPG key used to sign the GitLab packages was changed on April 6, 2020. If you had configured the GitLab package repositories on your machines prior to that, you will need to take some manual steps to add the new key before you can install or update GitLab packages. See the Omnibus Improvements for more details.
PostgreSQL 11 is the default version of PostgreSQL in GitLab 12.10. All new installs of GitLab will automatically default to PostgreSQL 11. For upgrades of existing GitLab installations, the PostgreSQL version will be automatically upgraded to PostgreSQL 11 if PostgreSQL 9.6 or 10 are detected. However, major PostgreSQL upgrades on installations with High Availability and/or Geo require some manual steps. If GitLab detects that you are using repmgr or Geo, it will ask you to follow the manual upgrade instructions for HA or for Geo; we will not automatically upgrade your PostgreSQL database. For 12.10, if you prefer to upgrade your GitLab version first and upgrade PostgreSQL later, you still have the option to stay on PostgreSQL 9.6 or 10 by disabling the PostgreSQL upgrade before you upgrade GitLab (sudo touch /etc/gitlab/disable-postgresql-upgrade). Note: There is a known issue when upgrading to 12.10 when using an external PostgreSQL database rather than the Omnibus-packaged database, resulting in the need to take some manual steps. To track this bug and see the workaround refer to the related issue.
Please note that PostgreSQL 9.6 and 10 will be removed in the next version of GitLab (13.0). This means that you will need to upgrade your PostgreSQL database to PostgreSQL 11 before you can upgrade to GitLab 13.0. We are very excited about the database performance improvements that PostgreSQL 11 will allow us to make and look forward to sharing the benefits of these improvements with you over the coming months.
Please check out the changelog to see all the named changes: