Step 1
Before you start
- You need permission to install GitHub Apps for the account or organization that owns the repository. Otherwise, GitHub will let you request approval from an owner.
- GitHub Actions must be enabled if you want CodeMD Impacted Tests to run.
- Choose whether CodeMD should access all repositories or only selected repositories. Selected repositories is the least-privilege choice.
- Ask whether organization rulesets already protect the repository before adding another one.
Recommended: Start with one non-production repository, verify the complete flow, and then add more repositories.
Step 2
Install the CodeMD.dev GitHub PR Checks App
Open the installation page
Select Install CodeMD on GitHub. GitHub opens the CodeMD.dev PR Checks App installation screen.
Choose the owner
Select the personal account or organization that owns the repository. If installations require approval, choose Request and send it to an organization owner.
Choose repositories
Select Only select repositories, then choose each repository where CodeMD should run. Avoid All repositories unless that access is intentional.
Review and approve
Review the permissions GitHub displays, then select Install. An owner may need to approve updated permissions before test runs can start.
Confirm access
Open the CodeMD PR Checks Dashboard, sign in with GitHub, and confirm the selected repository is listed.
Step 3
Understand the requested permissions
GitHub App permissions cover an entire repository. GitHub does not provide branch-scoped GitHub App permissions, so CodeMD requests repository-level write permission for test execution.
| Permission | Why it is needed | How CodeMD uses it |
|---|---|---|
| Contents: read and write | Read the checked commit and update the test branch. | Writes only to codemd-test-runs, never the default, source, feature, or pull-request branches. |
| Workflows: read and write | Put the CodeMD test workflow on the test branch. | Writes .github/workflows/codemd-tests.yml only on codemd-test-runs. |
| Actions: read | Watch the test run and read its result. | Reports pass, fail, and changed-line coverage. |
| Checks: read and write | Post results on commits and pull requests. | Creates and updates CodeMD check results. |
| Pull requests and metadata: read | Identify commits and changed files. | Reads pull-request context; it does not edit or merge pull requests. |
Defense in depth: The ruleset below is enforced by GitHub independently of CodeMD. It prevents the app from updating source branches even though GitHub grants Contents write at repository scope.
Step 4
Lock CodeMD out of source branches
This strong ruleset targets every branch except codemd-test-runs. Only the trusted people or teams on its bypass list can update protected branches. Never add CodeMD to that list.
Open repository rules
Open the repository and go to Settings → Rules → Rulesets → New ruleset → New branch ruleset.
Name and activate it
Name it Protect source branches from apps. Set enforcement to Active. If your plan offers Evaluate, you can review its effect first and activate it afterward.
Target all branches, with one exception
Under Target branches, include all branches. Add an exclusion for the exact branch codemd-test-runs. Without that exclusion, CodeMD cannot start tests.
Set the bypass list
Add only named, trusted people or teams that need to update ordinary branches. Add another trusted bot only if your workflow requires it. Do not add the CodeMD.dev PR Checks App. Avoid broad role-based bypasses when a named maintainer team will work.
Enable protections
- Enable Restrict updates.
- Enable Restrict deletions.
- Keep force pushes blocked.
- Optionally require pull requests, reviews, and status checks for critical branches.
Review, save, and test
Check the target preview and bypass list, then save. Verify that maintainers can still push feature branches and merge pull requests. Restricting all updates without the correct human/team bypass will also block legitimate work.
Inspect inherited rules
Review every active repository and organization ruleset. Rulesets combine, so an inherited rule that targets codemd-test-runs can still block tests.
Expected result: GitHub allows CodeMD to update codemd-test-runs, but rejects the app if it tries to update the default branch or another protected branch.
See GitHub's current documentation for creating repository rulesets and available rules and plan limits.
Step 5
Verify the complete setup
- Create a small branch and open a test pull request against the default branch.
- In the pull request's Checks area, confirm CodeMD Call-Path Security and CodeMD Impacted Tests appear.
- In the repository's Actions tab, confirm a CodeMD test run starts from
codemd-test-runs. - Under Code → Branches, confirm CodeMD updated only
codemd-test-runs. - Merge or close the test pull request using your normal process to confirm maintainers are not blocked.
- If available, check Settings → Rules → Insights for branch and actor activity.
Step 6
Troubleshooting
“Tests didn't run” or a permission error appears
An owner may not have accepted the current Contents, Workflows, and Actions permissions. Open GitHub App installations, choose CodeMD.dev PR Checks, and review pending permission requests.
No Actions run appears
Confirm Actions is enabled, codemd-test-runs is excluded from every restrictive ruleset, and organization policy permits the workflow.
The repository is missing
Open the app's Configure page and verify the repository is selected and the organization approved the installation.
Developers or merges are blocked
Restrict updates permits updates only from bypass actors. Add the appropriate trusted maintainer team or narrow protection to critical branches. Do not add CodeMD to the bypass list.
A required check never completes
Temporarily remove CodeMD from required status checks while diagnosing permissions or Actions policy. Restore it after a test pull request succeeds.
Step 7
Change or remove access
Open GitHub Settings → Applications → Installed GitHub Apps and select Configure for CodeMD.dev PR Checks.
- Edit Repository access to add or remove repositories.
- Select Uninstall to stop CodeMD entirely.
- After uninstalling, CodeMD cannot read the repository or post checks. See the Privacy Policy for deletion and retention details.
- You may delete
codemd-test-runsafter uninstalling if you do not need its history.