Configuring Policy Filter Map Size In Cilium And Tetragon

In the realm of cloud-native security and observability, Cilium and Tetragon stand out as powerful tools. Cilium provides network connectivity and security, while Tetragon enhances security observability with deep visibility into kernel-level activities. A crucial aspect of their functionality is the ability to enforce policies, and the size of the policy filter map plays a significant role in how many policies can be active simultaneously. This article delves into the importance of configuring the policy filter map size, the challenges posed by a restrictive size, and a proposed solution to make this configuration dynamic and scalable.

Understanding the Policy Filter Map Size

The policy filter map is a critical component within Tetragon's architecture. It's essentially a data structure, specifically a hash-of-maps, that stores the active policies. These policies dictate how Tetragon observes and reacts to system events. The size of this map directly limits the number of concurrent policies that can be enforced. A larger map allows for more policies, providing greater flexibility and granularity in security enforcement. However, a smaller map can become a bottleneck, especially in environments with complex policy requirements.

Currently, Tetragon has a fixed limit of 128 policies due to the POLICY_FILTER_MAX_POLICIES constant defined in its eBPF program:

#define POLICY_FILTER_MAX_POLICIES   128

struct {
	__uint(type, BPF_MAP_TYPE_HASH_OF_MAPS);
	__uint(max_entries, POLICY_FILTER_MAX_POLICIES);
	__type(key, u32); /* policy id */
	__array(
		values, struct {
			__uint(type, BPF_MAP_TYPE_HASH);
			__uint(max_entries, 1);
			__type(key, __u64); /* cgroup id */
			__type(value, __u8); /* empty  */
		});
} policy_filter_maps SEC(".maps");

This fixed limit can be restrictive for users operating with extensive policy sets. When the limit is reached, new policies may be dropped or rejected, leading to potential security gaps or unexpected behavior. This limitation underscores the need for a more flexible approach to managing the policy filter map size.

The Problem with a Restrictive Policy Filter Map Size

The limitation of 128 concurrent policies can present several challenges, particularly in dynamic and complex environments. Let's explore some scenarios where this restriction can become problematic:

• Large-Scale Deployments: In large deployments with numerous microservices and applications, the number of policies required to adequately secure and monitor the environment can easily exceed 128. Each microservice might require specific policies to control its behavior and interactions, and a fixed limit can hinder the ability to implement fine-grained security controls.
• Dynamic Policy Requirements: Modern applications often have dynamic policy requirements, where policies need to be added, modified, or removed based on changing conditions. For instance, policies might need to be adjusted during deployments, scaling events, or security incidents. A fixed policy limit makes it difficult to adapt to these dynamic needs, potentially leaving the system vulnerable during transitions.
• Complex Security Policies: Some security policies are inherently complex, requiring multiple rules and conditions to accurately capture the desired behavior. Implementing such policies can consume a significant portion of the available policy slots, leaving less room for other necessary policies. This can force administrators to make difficult trade-offs between different security requirements.
• Policy Evolution and Testing: As applications evolve and new security threats emerge, policies need to be updated and tested. A restrictive policy limit can make this process challenging, as it might be necessary to remove existing policies to make room for new ones. This can disrupt the ongoing security posture of the system and increase the risk of misconfigurations.

In essence, the fixed policy filter map size creates a bottleneck that limits the scalability and adaptability of Tetragon's policy enforcement capabilities. To address this issue, a more configurable and dynamic approach is required.

Proposed Solution: Dynamic Configuration of Policy Filter Map Size

To overcome the limitations of the fixed policy filter map size, a solution is proposed to make the map size configurable. This approach draws inspiration from how the execve_map is configured in Tetragon, providing a consistent and familiar mechanism for users.

The proposed solution involves the following key steps:

1. Replace the Fixed Limit in bpf/process/policy_filter.h: The current fixed limit defined by __uint(max_entries, POLICY_FILTER_MAX_POLICIES) in bpf/process/policy_filter.h will be replaced with __uint(max_entries, 1). This removes the static cap on the policy filter map size within the eBPF program.
2. Add a New Configuration Option: A new command-line option flag, policy-filter-map-entries, will be introduced. This flag will allow users to specify the desired maximum number of entries for the policy filter map. The implementation will mirror the existing approach used for the execve-map-entries option.
3. Default to Backward Compatibility: To ensure backward compatibility and minimize disruption for existing users, the default value for the policy-filter-map-entries option will be set to 128. This means that if users do not explicitly configure the map size, Tetragon will continue to behave as it does currently.
4. Agent-Side Configuration: The Tetragon agent will be responsible for setting the `spec.Maps[