> ## Documentation Index
> Fetch the complete documentation index at: https://semgrep-ee9d73d8-mintlify-b75b9a88.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Semgrep Secrets rule structure and sample

> This article walks you through writing, publishing, and using Semgrep Secrets rules. It also demonstrates what a sample Semgrep Secrets rule looks like, with subsequent sections describing the key-value pairs in the context of a Semgrep Secrets rule.

## Write a rule

There are two ways to write a rule for Semgrep Secrets:

1. Create a YAML file.
2. Use the Semgrep editor.

### Create a YAML file

If you're familiar with Semgrep's rules syntax, including the [validator syntax](/semgrep-secrets/validators), you can create a YAML file containing your rules. When you're done, [publish your rules for use with your organization](/writing-rules/private-rules/).

If you want to keep your rules file local, you must pass in the `--allow-untrusted-validators` flag when calling `semgrep ci` from the CLI.

### Use Semgrep Editor

The Semgrep Editor, available in Semgrep AppSec Platform, can help you write custom Semgrep Secrets rules. To pull up a sample rule that you can modify:

<Steps>
  <Step>
    Sign in to Semgrep AppSec Platform.
  </Step>

  <Step>
    Go to **Rules > Editor**.
  </Step>

  <Step>
    Click the **+** icon and, under **Secrets**, select **HTTP validators**.
  </Step>
</Steps>

Semgrep Editor allows you to modify the sample rule and run it against test code to ensure it functions as expected. When you finish making changes, click **Save** to proceed.

<Note>
  **INFO**

  Custom validator rules are private to your organization. They are not available to the Semgrep Community.
</Note>

To run a specific rule when invoking Semgrep from the CLI:

<Steps>
  <Step>
    Sign in to Semgrep AppSec Platform.
  </Step>

  <Step>
    Go to **Rules > Editor**.
  </Step>

  <Step>
    Open up your rule.
  </Step>

  <Step>
    Click **Add to** <Tooltip tip="A policy defines the set of rules that Semgrep runs and the workflow actions it undertakes when a rule from the policy generates a finding. The workflow action can include notifying Slack channels or posting a comment in the pull request or merge request that generated the finding. Not to be confused with policy-as-code." cta="See full definition." href="/semgrep-code/glossary#policy">Policy</Tooltip> and select your mode: Monitor, Comment, or Blocking.
  </Step>

  <Step>
    In the CLI, start a scan by running `semgrep ci`.
  </Step>
</Steps>

## Sample rule

The following sample rule detects a leaked GitHub personal access token (PAT):

```yaml expandable theme={null}
rules:
- id: github_example
  message: >-
    This is an example rule, that performs validation against github.com
  severity: MEDIUM
  languages:
  - regex
  validators:
  - http:
      request:
        headers:
          Authorization: Bearer $REGEX
          Host: api.github.com
          User-Agent: Semgrep
        method: GET
        url: https://api.github.com/user
      response:
      - match:
        - status-code: 200
        result:
          validity: valid
      - match:
        - status-code: 401
        result:
          validity: invalid
  patterns:
  - patterns:
    - pattern-regex: (?<REGEX>\b((ghp|gho|ghu|ghs|ghr|github_pat)_[a-zA-Z0-9_]{36,255})\b)
    - focus-metavariable: $REGEX
    - metavariable-analysis:
        analyzer: entropy
        metavariable: $REGEX
```

This can also be done in any Semgrep-supported language:

```yaml expandable theme={null}
rules:
- id: github_example
  message: >-
    This is an example rule that performs validation against github.com
  severity: MEDIUM
  languages:
  - javascript
  - typescript
  validators:
  - http:
      request:
        headers:
          Authorization: Bearer $REGEX
          Host: api.github.com
          User-Agent: Semgrep
        method: GET
        url: https://api.github.com/user
      response:
      - match:
        - status-code: 200
        result:
          validity: valid
      - match:
        - status-code: 401
        result:
          validity: invalid
  patterns:
  - patterns:
    - pattern: |
        "$R"
    - metavariable-pattern:
        metavariable: $R
        patterns:
          - pattern-regex: (?<REGEX>\b((ghp|gho|ghu|ghs|ghr|github_pat)_[a-zA-Z0-9_]{36,255})\b)
    - focus-metavariable: $REGEX
    - metavariable-analysis:
        analyzer: entropy
        metavariable: $REGEX
```

### Subkeys under the `metadata` key

These subkeys provide context to both you and other end-users, as well as to
Semgrep.

```yaml theme={null}
  ...
  metadata:
    ...
    secret_type: GitHub
    technology:
    - secrets
  ...
```

| Key           | Description                                                                                                                                                                                                                                                |
| ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `secret_type` | Defines the name of the service or the type of secret. When writing a custom validator, set this value to a descriptive name to help identify it when triaging secrets. Examples of secret types include "Slack," "Asana," and other common service names. |
| `technology`  | Set this to `secrets` to identify the rule as a Secrets rule.                                                                                                                                                                                              |

### Subkeys under the `patterns` key

These subkeys identify the token to analyze in a given match.

```yaml theme={null}
  ...
  patterns:
  ...
  - pattern-regex: (?<REGEX>\b((ghp|gho|ghu|ghs|ghr|github_pat)_[a-zA-Z0-9_]{36,255})\b)
  - focus-metavariable: $REGEX
  - metavariable-analysis:
      analyzer: entropy
      metavariable: $REGEX
  ..
```

| Key                     | Description                                                                                                                                                             |
| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `pattern-regex`         | Searches for a regular expression and assigns it to the named capture group regex, which is then used as \$REGEX.                                                       |
| `focus_metavariable`    | This key enables the rule to define a metavariable upon which Semgrep can perform further analysis, such as entropy analysis.                                           |
| `metavariable_analysis` | Under `metavariable_analysis`, you can define additional keys: `analyzer` and `metavariable`. These specify the kind of analysis Semgrep performs and on what variable. |

<Tip>
  **TIP**

  For more information, see the rule syntax for [<Icon icon="file-text" iconType="solid" /> Focus metavariable](/writing-rules/rule-syntax/#focus-metavariable).
</Tip>

### Subkeys under the `validators` and `http` keys

The `validators` key uses a list of keys to define the validator function. In
particular, the `http` key defines how the rule forms a request object and what
response is expected for valid and invalid states. Although some rules do not use a `validators` key, most Secrets rules use it.

```yaml theme={null}
  ...
  validators:
  - http:
      request:
        headers:
          Authorization: Bearer $REGEX
          Host: api.github.com
          User-Agent: Semgrep
        method: GET
        url: https://api.github.com/user
      response:
        - match:
          - status-code: '200'
          result:
            validity: valid
        - match:
          - status-code: '404'
          result:
            validity: invalid
```

| Key        | Description                                                                                                                                                                                                                                                                                                                       |
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `request`  | This key and its subkeys describe the request object and the URL to send the request object to.                                                                                                                                                                                                                                   |
| `response` | This key and its subkeys determine **validation status**. Semgrep Secrets identifies a validation status through HTTP status code **and** other key-value pairs. For example, a rule may require a 200 status code **and** a `"message": "ok"` in the response body for the matching secret to be considered **Confirmed valid**. |

<Tip>
  **TIP**

  See [<Icon icon="file-text" iconType="solid" /> Validators](/semgrep-secrets/validators) for more information.
</Tip>

## Metavariable binding

Semgrep Secrets can use metavariables. Metavariables allow Semgrep Secrets to reuse matched information from your code in its validators. An example of a metavariable is as follows:

<Frame>
  <iframe title="Message displays metavariable content" src="https://semgrep.dev/embed/editor?snippet=JDzRR" width="100%" height="432px" loading="lazy" frameBorder="0" />
</Frame>

When you click **Run**, the content from the metavariable `$HELLO` displays as `This content is now reusable in validators`. If this were a Secrets rule, Semgrep Secrets could use this to call the appropriate service to determine if the secret is active.

## Differences between Semgrep Secrets rules and Semgrep Registry rules

The Semgrep Registry includes SAST rules that can detect secrets to a certain
extent. You can run these rules in Semgrep Code (Semgrep's SAST analyzer), or
even write your own custom secret-detecting SAST rules, but with the following
differences:

* Semgrep Code does not run a validator function against these rules, resulting in less accurate results.
  * Because the results are less accurate, these rules are not suitable as criteria to block a PR or MR.
* The UI for Semgrep Code is tailored to SAST triage and does not include filtering functions for valid or invalid tokens.
* Existing Semgrep Pro rules that detect secrets are transitioning from Semgrep Code to Semgrep Secrets. By transitioning these rules, improvements, such as validator functions, can be added to the rules when they are run in Semgrep Secrets.
* You can write your own custom validator functions and run them in Semgrep Secrets for custom services or use cases.
