Customize pipeline secret detection

include:
  - template: Jobs/Secret-Detection.gitlab-ci.yml

secret_detection:
  variables:
    SECRETS_ANALYZER_VERSION: "4.5"

include:
  - template: Jobs/Secret-Detection.gitlab-ci.yml

secret_detection:
  variables:
    SECRET_DETECTION_HISTORIC_SCAN: "true"

[secrets]
  [[secrets.ruleset]]
    disable = true
    [secrets.ruleset.identifier]
      type  = "gitleaks_rule_id"
      value = "RSA private key"

[secrets]
  [[secrets.ruleset]]
    [secrets.ruleset.identifier]
      type  = "gitleaks_rule_id"
      value = "RSA private key"
    [secrets.ruleset.override]
      description = "OVERRIDDEN description"
      message     = "OVERRIDDEN message"
      name        = "OVERRIDDEN name"
      severity    = "Info"

include:
  - template: Jobs/Secret-Detection.gitlab-ci.yml

variables:
  SECRET_DETECTION_RULESET_GIT_REFERENCE: "gitlab.com/example-group/remote-ruleset-project"

<AUTH_USER>:<AUTH_PASSWORD>@<PROJECT_PATH>@<GIT_SHA>

include:
  - template: Jobs/Secret-Detection.gitlab-ci.yml

variables:
  SECRET_DETECTION_RULESET_GIT_REFERENCE: "group_2504721_bot_7c9311ffb83f2850e794d478ccee36f5:$GROUP_ACCESS_TOKEN@gitlab.com/example-group/remote-ruleset-project"

[secrets]
  [[secrets.passthrough]]
    type   = "raw"
    target = "gitleaks.toml"
    value  = """
title = "replace default ruleset with a raw passthrough"

[[rules]]
description = "Test for Raw Custom Rulesets"
regex = '''Custom Raw Ruleset T[est]{3}'''
"""

[secrets]
  [[secrets.passthrough]]
    type   = "file"
    target = "gitleaks.toml"
    value  = "config/gitleaks.toml"

# .gitlab/secret-detection-ruleset.toml in https://gitlab.com/user_group/basic_repository
[secrets]
  [[secrets.passthrough]]
    type   = "git"
    ref    = "main"
    subdir = "config"
    value  = "https://gitlab.com/user_group/central_repository_with_shared_ruleset"

include:
  - template: Jobs/Secret-Detection.gitlab-ci.yml

secret_detection:
  variables:
    SECRET_DETECTION_HISTORIC_SCAN: "true"
```0

In this configuration the analyzer loads the ruleset configuration from `gitleaks.toml` file stored at the address provided.

For more information on the passthrough syntax to use, see [Schema](../pipeline/custom_rulesets_schema.md#schema).

#### With a private remote ruleset

If a ruleset configuration is stored in a private repository you must provide the credentials to access the repository by using the passthrough's [`auth` setting](../pipeline/custom_rulesets_schema.md#the-secretspassthrough-section).

The `auth` setting only works with `git` passthrough.

To use a remote ruleset stored in a private repository, add the following to the `.gitlab/secret-detection-ruleset.toml` configuration file stored in a repository, adjust the `value` to point to the address of the Git repository, and update `auth` to use the appropriate credentials:

```yaml
include:
  - template: Jobs/Secret-Detection.gitlab-ci.yml

secret_detection:
  variables:
    SECRET_DETECTION_HISTORIC_SCAN: "true"
```1

Beware of leaking credentials when using this feature. Check [this section](../pipeline/custom_rulesets_schema.md#interpolate) for an example on how to use environment variables to minimize the risk.

For more information on the passthrough syntax to use, see [Schema](../pipeline/custom_rulesets_schema.md#schema).

### Extend the default ruleset

You can also extend the [default ruleset](../detected_secrets.md) configuration with additional rules as appropriate. This can be helpful when you would still like to benefit from the high-confidence predefined rules maintained by GitLab in the default ruleset, but also want to add rules for types of secrets that may be used in your own projects and namespaces. New rules must follow the [custom rule format](custom_rulesets_schema.md#custom-rule-format).

#### With a local ruleset

You can use a `file` passthrough to extend the default ruleset to add additional rules.

Add the following to the `.gitlab/secret-detection-ruleset.toml` configuration file stored in the same repository, and adjust the `value` as appropriate to point to the path of the extended configuration file:

```yaml
include:
  - template: Jobs/Secret-Detection.gitlab-ci.yml

secret_detection:
  variables:
    SECRET_DETECTION_HISTORIC_SCAN: "true"
```2

The extended configuration stored in `extended-gitleaks-config.toml` is included in the configuration used by the analyzer
in the CI/CD pipeline.

In the example below, we add a couple of new `[[rules]]` sections that define a number of regular expressions to be detected:

```yaml
include:
  - template: Jobs/Secret-Detection.gitlab-ci.yml

secret_detection:
  variables:
    SECRET_DETECTION_HISTORIC_SCAN: "true"
```3

With this ruleset configuration the analyzer detects any strings matching with those two defined regex patterns.

For more information on the passthrough syntax to use, see [Schema](../pipeline/custom_rulesets_schema.md#schema).

#### With a remote ruleset

Similar to how you can replace the default ruleset with a remote ruleset, you can also extend the default ruleset with configuration stored in a remote Git repository or file stored external to the repository in which you have the `.gitlab/secret-detection-ruleset.toml` configuration file.

This can be achieved by using either of the `git` or `url` passthroughs as discussed previously.

To do that with a `git` passthrough, add the following to `.gitlab/secret-detection-ruleset.toml` configuration file stored in the same repository, and adjust the `value`, `ref`, and `subdir` as appropriate to point to the path of the extended configuration file:

```yaml
include:
  - template: Jobs/Secret-Detection.gitlab-ci.yml

secret_detection:
  variables:
    SECRET_DETECTION_HISTORIC_SCAN: "true"
```4

Pipeline secret detection assumes the remote ruleset configuration file is called `gitleaks.toml`, and is stored in `config` directory on the `main` branch of the referenced repository.

To extend the default ruleset, the `gitleaks.toml` file should use `[extend]` directive similar to the example above:

```yaml
include:
  - template: Jobs/Secret-Detection.gitlab-ci.yml

secret_detection:
  variables:
    SECRET_DETECTION_HISTORIC_SCAN: "true"
```5

To use a `url` passthrough, add the following to `.gitlab/secret-detection-ruleset.toml` configuration file stored in the same repository, and adjust the `value` as appropriate to point to the path of the extended configuration file

```yaml
include:
  - template: Jobs/Secret-Detection.gitlab-ci.yml

secret_detection:
  variables:
    SECRET_DETECTION_HISTORIC_SCAN: "true"
```6

For more information on the passthrough syntax to use, see [Schema](../pipeline/custom_rulesets_schema.md#schema).

### Ignore patterns and paths

There may be situations in which you need to ignore a certain pattern or path from being detected by pipeline secret detection. For example, you may have a file including fake secrets to be used in a test suite.

In that case, you can utilize [Gitleaks' native `[allowlist]`](https://github.com/gitleaks/gitleaks#configuration) directive to ignore specific patterns or paths.

This feature works regardless of whether you're using a local or a remote ruleset configuration file. The examples below utilizes a local ruleset using `file` passthrough though.

To ignore a pattern, add the following to the `.gitlab/secret-detection-ruleset.toml` configuration file stored in the same repository, and adjust the `value` as appropriate to point to the path of the extended configuration file:

```yaml
include:
  - template: Jobs/Secret-Detection.gitlab-ci.yml

secret_detection:
  variables:
    SECRET_DETECTION_HISTORIC_SCAN: "true"
```7

The extended configuration stored in `extended-gitleaks-config.toml` will be included in the configuration used by the analyzer.

In the example below, we add an `[allowlist]` directive that defines a regex that matches the secret to be ignored ("allowed"):

```yaml
include:
  - template: Jobs/Secret-Detection.gitlab-ci.yml

secret_detection:
  variables:
    SECRET_DETECTION_HISTORIC_SCAN: "true"
```8

This ignores any string matching `glpat-` with a suffix of 20 characters of digits and letters.

Similarly, you can exclude specific paths from being scanned. In the example below, we define an array of paths to ignore under the `[allowlist]` directive. A path could either be a regular expression, or a specific file path:

```yaml
include:
  - template: Jobs/Secret-Detection.gitlab-ci.yml

secret_detection:
  variables:
    SECRET_DETECTION_HISTORIC_SCAN: "true"
```9

This ignores any secrets detected in either `/gitleaks.toml` file or any file ending with one of the specified extensions.

For more information on the passthrough syntax to use, see [Schema](../pipeline/custom_rulesets_schema.md#schema).

### Ignore secrets inline

In some instances, you might want to ignore a secret inline. For example, you may have a fake secret in an example or a test suite. In these instances, you will want to ignore the secret instead of having it reported as a vulnerability.

To ignore a secret, add `gitleaks:allow` as a comment to the line that contains the secret.

For example:

```toml
[secrets]
  [[secrets.ruleset]]
    disable = true
    [secrets.ruleset.identifier]
      type  = "gitleaks_rule_id"
      value = "RSA private key"
```0

### Detecting complex strings

The [default ruleset](_index.md#detected-secrets) provides patterns to detect structured strings with a low rate of false positives.
However, you might want to detect more complex strings like passwords. Because [Gitleaks doesn't support lookahead or lookbehind](https://github.com/google/re2/issues/411),
writing a high-confidence, general rule to detect unstructured strings is not possible.

Although you can't detect every complex string, you can extend your ruleset to meet specific use cases.

For example, this rule modifies the [`generic-api-key` rule](https://github.com/gitleaks/gitleaks/blob/4e43d1109303568509596ef5ef576fbdc0509891/config/gitleaks.toml#L507-L514) from the Gitleaks default ruleset:

```toml
[secrets]
  [[secrets.ruleset]]
    disable = true
    [secrets.ruleset.identifier]
      type  = "gitleaks_rule_id"
      value = "RSA private key"
```1

This regular expression matches:

1. A case-insensitive identifier that starts with `pwd`, or `passwd` or `password`. You can adjust this with other variations like `secret` or `key`.
1. A suffix that follows the identifier. The suffix is a combination of digits, letters, and symbols, and is between zero and 23 characters long.
1. Commonly used assignment operators, like `=`, `:=`, `:`, or `=>`.
1. A secret prefix, often used as a boundary to help with detecting the secret.
1. A string of digits, letters, and symbols, which is between three and 50 characters long. This is the secret itself. If you expect longer strings, you can adjust the length.
1. A secret suffix, often used as a boundary. This matches common endings like ticks, line breaks, and new lines.

Here are example strings which are matched by this regular expression:

```toml
[secrets]
  [[secrets.ruleset]]
    disable = true
    [secrets.ruleset.identifier]
      type  = "gitleaks_rule_id"
      value = "RSA private key"
```2

To use this regex, extend your ruleset with one of the methods documented on this page.

For example, imagine you wish to extend the default ruleset [with a local ruleset](#with-a-local-ruleset-1) that includes this rule.

Add the following to a `.gitlab/secret-detection-ruleset.toml` configuration file stored in the same repository. Adjust the `value` to point to the path of the extended configuration file:

```toml
[secrets]
  [[secrets.ruleset]]
    disable = true
    [secrets.ruleset.identifier]
      type  = "gitleaks_rule_id"
      value = "RSA private key"
```3

In `extended-gitleaks-config.toml` file, add a new `[[rules]]` section with the regular expression you want to use:

```toml
[secrets]
  [[secrets.ruleset]]
    disable = true
    [secrets.ruleset.identifier]
      type  = "gitleaks_rule_id"
      value = "RSA private key"
```4

This example configuration is provided only for convenience, and might not work
for all use cases. If you configure your ruleset to detect complex strings, you might
create a large number of false positives, or fail to capture certain patterns.

## Offline configuration

- Tier: Premium, Ultimate
- Offering: GitLab Self-Managed

An offline environment has limited, restricted, or intermittent access to external resources through
the internet. For instances in such an environment, pipeline secret detection requires
some configuration changes. The instructions in this section must be completed together with the
instructions detailed in [offline environments](../../offline_deployments/_index.md).

### Configure GitLab Runner

By default, a runner tries to pull Docker images from the GitLab container registry even if a local
copy is available. You should use this default setting, to ensure Docker images remain current.
However, if no network connectivity is available, you must change the default GitLab Runner
`pull_policy` variable.

Configure the GitLab Runner CI/CD variable `pull_policy` to
[`if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy).

### Use local pipeline secret detection analyzer image

Use a local pipeline secret detection analyzer image if you want to obtain the image from a local Docker
registry instead of the GitLab container registry.

Prerequisites:

- Importing Docker images into a local offline Docker registry depends on your
  network security policy. Consult your IT staff to find an accepted and approved process
  to import or temporarily access external resources.

1. Import the default pipeline secret detection analyzer image from `registry.gitlab.com` into your
   [local Docker container registry](../../../packages/container_registry/_index.md):

   ```plaintext
   registry.gitlab.com/security-products/secrets:6