Development Edit

OPA is written in the Go programming language.

If you are not familiar with Go we recommend you read through the How to Write Go Code article to familiarize yourself with the standard Go development environment.

Requirements:

  • Git
  • GitHub account (if you are contributing)
  • Go (version 1.15+ is supported though older versions are likely to work)
  • GNU Make

Getting Started

After cloning the repository, just run make. This will:

  • Build the OPA binary.
  • Run all of the tests.
  • Run all of the static analysis checks.

If the build was successful, a binary will be produced in the top directory (opa_<OS>_<ARCH>).

Verify the build was successful with ./opa_<OS>_<ARCH> run.

You can re-build the project with make build, execute all of the tests with make test, and execute all of the performance benchmarks with make perf.

The static analysis checks (e.g., go fmt, golint, go vet) can be run with make check.

To correct any imports or style errors run make fmt.

Workflow

Fork, clone, create a branch

Go to https://github.com/open-policy-agent/opa and fork the repository into your account by clicking the “Fork” button.

Clone the fork to your local machine:

git clone git@github.com/<GITHUB USERNAME>/opa.git opa
cd opa
git remote add upstream https://github.com/open-policy-agent/opa.git

Create a branch for your changes.

git checkout -b somefeature

Developing your change

Develop your changes and regularly update your local branch against upstream, for example by rebasing:

git fetch upstream
git rebase upstream/main

Be sure to run make check before submitting your pull request. You may need to run go fmt on your code to make it comply with standard Go style.

Submission

Commit changes and push to your fork.

git commit -s
git push origin somefeature

Make sure to use a good commit message.

Now, submit a Pull Request from your fork. See the official GitHub Documentation for instructions to create the request.

Hint: You should be prompted to with a “Compare and Pull Request” button that mentions your new branch on https://github.com/open-policy-agent/opa

Once your Pull Request has been reviewed and signed off please squash your commits. If you have a specific reason to leave multiple commits in the Pull Request, please mention it in the discussion.

If you are not familiar with squashing commits, see the following blog post for a good overview.

Built-in Functions

Built-in Functions can be added inside the topdown package in this repository.

Built-in functions may be upstreamed if they are generally useful and provide functionality that would be impractical to implement natively in Rego (e.g., CIDR arithmetic). Implementations should avoid thirdparty dependencies. If absolutely necessary, consider importing the code manually into the internal package.

All built-in function implementations must include a test suite. See test/cases/testdata/helloworld in this repository for an example of how to implement tests for your built-in functions.

Benchmarks

Several packages in this repository implement benchmark tests. To execute the benchmarks you can run make perf in the top-level directory. We use the Go benchmarking framework for all benchmarks. The benchmarks run on every pull request.

To help catch performance regressions we also run a batch job that compares the benchmark results from the tip of main against the last major release. All of the results are posted and can be viewed here.

Dependencies

OPA is a Go module https://github.com/golang/go/wiki/Modules and dependencies are tracked with the standard go.mod file.

We also keep a full copy of the dependencies in the vendor directory. All go commands from the Makefile will enable module mode by setting GO111MODULE=on GOFLAGS=-mod=vendor which will also force using the vendor directory.

To update a dependency ensure that GO111MODULE is either on, or the repository qualifies for auto to enable module mode. Then simply use go get .. to get the version desired. This should update the go.mod and (potentially) go.sum files. After this you MUST run go mod vendor to ensure that the vendor directory is in sync.

Example workflow for updating a dependency:

go get -u github.com/sirupsen/logrus@v1.4.2  # Get the specified version of the package.
go mod tidy                                  # (Somewhat optional) Prunes removed dependencies.
go mod vendor                                # Ensure the vendor directory is up to date.

If dependencies have been removed ensure to run go mod tidy to clean them up.

Tool Dependencies

Sometimes we use some tools which are versioned and vendored with OPA as dependencies. For now, we have none, but any we use in the future should go in tools.go.

More details on the pattern: https://github.com/go-modules-by-example/index/blob/master/010_tools/README.md

Update these the same way as any other Go package. Ensure that any build script only uses go run ./vendor/<tool pkg> to force using the correct version.

Go

If you need to update the version of Go used to build OPA you must update these files in the root of this repository:

  • .go-version- which is used by the Makefile and CI tooling. Put the exact go version that OPA should use.

CI Configuration

OPA uses Github Actions defined in the .github/workflows directory.

Github Action Secrets

The following secrets are used by the Github Action workflows:

NameDescription
S3_RELEASE_BUCKETAWS S3 Bucket name to upload edge release binaries to. Optional – If not provided the release upload steps are skipped.
AWS_ACCESS_KEY_IDAWS credentials required to upload to the configured S3_RELEASE_BUCKET. Optional – If not provided the release upload steps are skipped.
AWS_SECRET_ACCESS_KEYAWS credentials required to upload to the configured S3_RELEASE_BUCKET. Optional – If not provided the release upload steps are skipped.
DOCKER_IMAGEFull docker image name (with org) to tag and publish OPA images. Optional – If not provided the image defaults to openpolicyagent/opa.
DOCKER_WASM_BUILDER_IMAGEFull docker image name (with org) to tag and publish WASM builder images. Optional – If not provided the image defaults to openpolicyagent/opa-wasm-builder.
DOCKER_USERDocker username for uploading release images. Will be used with docker login. Optional – If not provided the image push steps are skipped.
DOCKER_PASSWORDDocker password or API token for the configured DOCKER_USER. Will be used with docker login. Optional – If not provided the image push steps are skipped.
SLACK_NOTIFICATION_WEBHOOKSlack webhook for sending notifications. Optional – If not provided the notification steps are skipped.
TELEMETRY_URLURL to inject at build-time for OPA version reporting. Optional – If not provided the default value in OPA’s source is used.
NETLIFY_BUILD_HOOK_URLURL to trigger Netlify (openpolicyagent.org) deploys after release. Optional – If not provided the Netlify steps are skipped.

Periodic Workflows

Some of the Github Action workflows are triggered on a schedule, and not included in the post-merge, pull-request, etc actions. These are reserved for time consuming or potentially non-deterministic jobs (race detection tests, fuzzing, etc).

Below is a list of workflows and links to their status:

WorkflowDescription
NightlyRuns once per day at 8:00 UTC.