TL;DR: This tutorial will show you how to re-write a replicaset-controller (kube-controller-manager) in Java. This tutorial requires you to have a basic knowledge of spring-boot framework and maven commands and you're will be able write your own custom Java kubernetes controller similarly after reading this document. The complete example project is available at yue9944882/replicaset-controller and a working image example is also at ghcr.io/yue9944882/java-replicaset-controller.

Please check that you have the following required development toolings installed in your local environment:

• JDK 8+: Necessary Java development environment.
• Maven: Greater than 3.0.0+.
• KinD: (Optional) For verifying the controller in a real kubernetes cluster. https://kind.sigs.k8s.io/docs/user/quick-start/

Creating a new repo following the standard maven project structure, and ensure the following plugins are present in the pom.xml:

• Maven Compiler Plugin: Setting language level to 8.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <configuration>
        <source>8</source>
        <target>8</target>
    </configuration>
</plugin>

• Spring Boot Maven Plugin: For building thinner docker images following the standard spring-boot plugins:

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <version>2.3.4.RELEASE</version>
</plugin>

Adding the following dependency to your pom.xml.

<dependency>
	<groupId>io.kubernetes</groupId>
	<artifactId>client-java-spring-integration</artifactId>
	<version>${ >= 11.0.0 recommended}</version>
</dependency>

The dependency will auto-configure the necessary configuration beans for injecting informers and reconcilers to your application. And note that you can disable the configuration beans by setting the following properties in your spring context:

kubernetes.informer.enabled=false   # disables informer injection
kubernetes.reconciler.enabled=false # disables reconciler injection

So that the project can be packaged as a executable jar. See the example class here.

Adding the following annotation to whatever Java class under spring context, so that processors can be activated.

@ComponentScan("io.kubernetes.client.spring.extended.controller")

Corresponding example is available here.

You're supposed to create a new class (can be a inner-class) extending io.kubernetes.client.informer.SharedInformerFactory, and then put @KubernetesInformers annotation on the new class to configure the informer-factory list-watching the specified kubernetes resources. See the example at here. The following code is an example of registering a pod-informer and replicaset-informer to the informer-factory.

@KubernetesInformers({
        @KubernetesInformer( // Adding a pod-informer to the factory for list-watching pod resources
                apiTypeClass = V1Pod.class,
                apiListTypeClass = V1PodList.class,
                groupVersionResource =
                @GroupVersionResource(
                        apiGroup = "",
                        apiVersion = "v1",
                        resourcePlural = "pods")),
        @KubernetesInformer( // Adding a replicaset-informer to the factory for list-watching replicaset resources
                apiTypeClass = V1ReplicaSet.class,
                apiListTypeClass = V1ReplicaSetList.class,
                groupVersionResource =
                @GroupVersionResource(
                        apiGroup = "apps",
                        apiVersion = "v1",
                        resourcePlural = "replicasets")),
})
class ControllerSharedInformerFactory extends SharedInformerFactory {
}

And don't forget to register ControllerSharedInformerFactory as a bean to the spring context. See here. The registered informer-factory won't be running unless you explcitly calls startAllRegisteredInformers somewhere else, the method is the trigger to run the controller, so hold it carefully until you're ready :). In the example project, the informer-factory was started inside ControllerManager#run.

As a deeper insight, it's the KubernetesInformerFactoryProcessor parsing the @KubernetesInformers annotation on informer-factory class and then register SharedInformer and Lister beans for the kubernetes resource classes. You can easily acquire them by @Autowired (for >= 11.0.0 release) or declaring them as parameters in the bean method. See this example source to see the per-resource informer/lister bean registration here.

Now the informer-factory's ready, it will keep receiving watching events from kubernetes cluster after started. And by this step you will be actually writing a controller subscribing events from the cluster. First of all, create a new class implementing the Reconciler interface and register the class as a bean in the spring context:

public class ReplicaSetReconciler implements Reconciler {...}

Then add a @KubernetesReconciler annotation on the reconciler class to explictly mark the class so that it can be processed by KubernetesReconcilerConfigurer. The processor will be wiring the reconciler class to the informer-factory together so that the event can be delivered to the reconciler. In the @KubernetesReconciler annotation, you're also requried to uniquely name the reconciler by setting value property and set @KubernetesReconcilerWatches property to set what kind of kubernetes resources the reconciler need to keep notified. Note that the @KubernetesReconcilerWatches configuration won't raise a new watch connection, all the watch connections are managed by informer-factory and they're multiplex'd. Simply put, it's justing adding an in-memory watcher to the watch connection. For more detail, take a look at the example code here.

You can easily acquire SharedInformer and Lister instances via @Autowired annotations instead of passing them from the bean constructor, see the following example:

public class ReplicaSetReconciler implements Reconciler {
    @Autowired private SharedInformer<V1Pod> podInformer;
    @Autowired private Lister<V1Pod> podLister;
    @Autowired private SharedInformer<V1Node> nodeInformer;
    @Autowired private Lister<V1Node> nodeLister;
    ...
}

Note that the type parameter for the informer and lister i.e. the kubernetes api resource type must be declared in your @KubernetesInformers annotation.

Both the event-filter method and ready-func method is supposed to be "public" access.

public class ReplicaSetReconciler implements Reconciler {
    ...
    @AddWatchEventFilter(apiTypeClass = V1Pod.class)
    public boolean onAddFilter(V1Pod pod) {
      return true; // returns true to handle the event
    }
    
    @UpdateWatchEventFilter(apiTypeClass = V1Pod.class)
    public boolean onUpdateFilter(V1Pod oldPod, V1Pod newPod) {
      return true; // returns true to handle the event
    }
    
    @DeleteWatchEventFilter(apiTypeClass = V1Pod.class)
    public boolean onDeleteFilter(V1Pod pod) {
      return true; // returns true to handle the event
    }
    
    @KubernetesReconcilerReadyFunc
    public boolean podInformerCacheReady() {
      return podInformer.hasSynced(); // return true if reconciler is ready to run.
    }
    ..
}

The controller bean is the entry bean to run your reconciler class definition. You're supposed to create a controller bean instance using KubernetesControllerFactory:

@Configuration
public class MyConfiguration {
    ...
    @Bean
    public KubernetesControllerFactory replicasetController(
        SharedInformerFactory sharedInformerFactory,
        Reconciler reconciler) {
      return new KubernetesControllerFactory(sharedInformerFactory, reconciler);
    }
    ...
}

Now both the informer-factory and the reconciler are set, the last step is to stitch them together by adding a starter (or a CommandLineRunner) or a bean implemeting InitializingBean as is shown in the following approaches:

• (Option 1) Run it immediately.

@Component
public class ControllerStarter implements InitializingBean {

    @Resource
    private SharedInformerFactory sharedInformerFactory;

    @Resource(name = "replicaset-reconciler")
    private Controller replicasetController;

    @Override
    public void afterPropertiesSet() throws Exception {
        sharedInformerFactory.startAllRegisteredInformers();
        Executors.newSingleThreadExecutor().submit(replicasetController);
    }
}

• (Option 2) Pack the informer-factory and the reconciler into a controller-manager instance.

@Component
public class ControllerStarter implements InitializingBean {

    @Resource
    private SharedInformerFactory sharedInformerFactory;

    @Resource(name = "replicaset-reconciler")
    private Controller replicasetController;

    @Override
    public void afterPropertiesSet() throws Exception {
        ControllerManager controllerManager = new ControllerManager(sharedInformerFactory, replicasetController);
        Executors.newSingleThreadExecutor().submit(controllerManager);
    }
}

• (Option 3) Pack the informer-factory and the reconciler into a leader-election HA controller-manager.

@Component
public class ControllerStarter implements InitializingBean {

    @Resource
    private SharedInformerFactory sharedInformerFactory;

    @Resource(name = "replicaset-reconciler")
    private Controller replicasetController;

    @Override
    public void afterPropertiesSet() throws Exception {
        ControllerManager controllerManager = new ControllerManager(sharedInformerFactory, replicasetController);

        String appNamespace = "default";
        String appName = "java-replicaset-controller";
        String lockHolderIdentityName = UUID.randomUUID().toString(); // Anything unique
        EndpointsLock lock = new EndpointsLock(appNamespace, appName, lockHolderIdentityName);

        LeaderElectionConfig leaderElectionConfig =
            new LeaderElectionConfig(
                lock, Duration.ofMillis(10000), Duration.ofMillis(8000), Duration.ofMillis(2000));
        ExecutorService pool = Executors.newSingleThreadExecutor();
        try (LeaderElector leaderElector = new LeaderElector(leaderElectionConfig)) {
          leaderElector.run(
              () -> pool.submit(controllerManager),
              () -> pool.shutdown());
        }
    }
}

Run the command to build a thinner image for your controller:

mvn spring-boot:build-image

For running the image in a real cluster, please following the instructions here.