Translate
Enforcing OPA policies in another context
While OPA is able to perform evaluation solely based on (pre)defined policies and user input, certain evaluations require additional "context" from the caller to complete. In particular, this applies to search operations, where the objective is to enumerate all documents that satisfy a particular policy.
The straightforward implementation in this scenario would be to retrieve all documents from e.g. the database, and then pass this context to OPA as part of the user input. However, this is not always feasible, as the number of documents can be large. Furthermore, it is clearly inefficient if the caller is interested in the first few entries, as is the case with pagination.
The ideal solution would be to somehow "map" the OPA policies to a form that can be evaluated by the database itself -- ElasticSearch queries for example. OPA addresses this use case via the Compile API. By providing the query, user input, and the unknowns, OPA can return the AST of a simplified version of the policy. The Policy Service takes this one step further and translates the AST into the native database query. The caller is then responsible to take the translated query and execute it against the data source.
Currently, only translation to ElasticSearch queries is supported. Other targets, such as SQL, may be supported in a future release.
Example
In the default configuration, the API is exposed at /api/policy/v1/translate. As an example, assume that there is a predefined OPA policy search.rego, and it is desired to have all ElasticSearch queries retrieve only documents that pass this policy.
Policy Definition
This policy says that, the data.search.allow rule will return true if any one of the members in the acl.viewers array equals any one of the members in user.groups array. While input is a designated global variable in OPA that refers to the input, the record and user variables are arbitrary and is up to the caller to define in the policy.
package search
allow {
input.record.acl.viewers[_] == input.user.groups[_]
}
Additional information on policies, check OPA policy reference.
Translation Payload
The caller can then POST a request body:
{
"query": "data.search.allow == true",
"input": {
"user": {
"groups": ["sample-group-1"],
},
},
"unknowns": ["input.record"]
}
Translation Interpretation
The above request body asks Policy Service (OPA) the following question:
Given the user profile containing group membership information, what query should be performed to fill up the variable input.record, such that the policy data.search.allow evaluates to true?
How Policy Service answers it:
- Internally, the Policy Service forwards the information to OPA for the partial evaluation process and gets back an AST (Abstract Syntax Tree).
- The Policy Service then translates the AST into ElasticSearch queries and returns it to the caller.
Translation API Response
The output should look something like this:
{
"query": {
"bool": {
"should": [{
"bool": {
"filter": [{
"term": {
"acl.viewers": "sample-group-1"
}
}]
}
}]
}
}
}
It basically says to filter and return only documents that have sample-group-1 in the acl.viewers array (it is assumed that the field has a keyword type family). The caller can then take this query, append it to the original ElasticSearch query, and finally evaluate it against ElasticSearch.
Improvements introduced in M18 release
The translate logic prior to M18 could only handle simple allow policies. In order to handle policies containing deny rules and complex data evaluation logic like http calls, the preprocessor concept was introduced for the translate API.
The policy writer can optionally specify the preprocessor_config object in the policy as in the following example:
package osdu.partition["osdu"].search
import data.osdu.partition["osdu"].search_preprocessor
default allow := false
default deny := false
preprocess_config := {
"input_from_preprocessor": {
"allow_groups": search_preprocessor.allow_groups,
"deny_groups": search_preprocessor.deny_groups
},
"has_allow_rule": true,
"has_deny_rule": true
}
allow {
input.operation == "view"
input.record.acl.owners[_] == input.allow_groups[_]
}
deny {
input.operation == "view"
input.record.acl.owners[_] == input.deny_groups[_]
}
Preprocessor Config Attributes
The preprocessor_config has four attributes:
has_allow_ruletells the translate api if the policy contains any allow rules or not.has_deny_ruletells the translate api if the policy contains any deny rules or not.input_from_preprocessorcontains the data that can be evaluated by the OPA data api which is called first inside the translate api.es_subqueryallow the search policy to construct its own elastic search subquery,es_subquery: {"query": {...}}. This feature was added in M20.
The evaluated data in 'input_from_preprocessor' will be included in the input data for translating the allow/deny rules to ElasticSearch subqueries. The allow rules and the deny rules are translated separately following the same logic described at the beginning of this document. The combined subquery from the allow rules and the deny rules' translation results is returned from the translate api.
Example Policy
For the example above, combined with the following 'search_preprocessor' policy module:
package osdu.partition["osdu"].search_preprocessor
allow_groups := ["allow_group1", "allow_group2"]
deny_groups := ["deny_group1", "deny_group2"]
Translation API Payload
and with the following payload posted to the translate api:
{
"query": "osdu.partition["osdu"].search.allow == true",
"input": {
"operation" == "view"
},
"unknowns": ["input.record"]
}
Translation API Response
The translate api will return the following ElasticSearch subquery:
{
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"bool": {
"filter": [
{
"terms": {
"acl.owners": [
"allow_group1",
"allow_group2"
]
}
}
]
}
}
]
}
},
{
"bool": {
"must_not": [
{
"bool": {
"should": [
{
"bool": {
"filter": [
{
"terms": {
"acl.owners": [
"deny_group1",
"deny_group2"
]
}
}
]
}
}
]
}
}
]
}
}
]
}
}
}
For more information on translate API.