Deployment of Apache Ignite nodes in the form of Kubernetes pods has certain requirements pertaining to Apache Ignite's nodes discovery mechanism. It is likely that you won't be able to use the straightforward multicast based IP finder because multicast traffic is usually blocked by firewalls. However, you can use the static IP finder and list the IP addresses of all the Ignite nodes as long as Kubernetes assigns the addresses dynamically.
There is a chance that Amazon AWS IP finder, Google Compute Engine IP finder or JClouds IP finder will suit your needs but Kubernetes has to be deployed in one of these cloud environments. Alternatively, a shared file system or a relational database can be used for the sake of Apache Ignite nodes auto-discovery but you will have to maintain the database or the shared file system separately.
This documentation provides information about the IP finder developed specifically for cases when Apache Ignite nodes are containerized by Kubernetes. The IP finder automatically lookups IP addresses of all the alive Ignite pods by talking to a distinct Kubernetes service that keeps all the endpoints up to date.
Kubernetes Deployment Getting Started Guide
Refer to Kubernetes Deployment Getting Started Guide that will walk you through a complete set of actions that need to be done in order to deploy an Ignite cluster in Kubernetes.
To enable nodes auto-discovery in Kubernetes,
TcpDiscoveryKubernetesIpFinder needs to be set in the configuration as shown below:
<bean class="org.apache.ignite.configuration.IgniteConfiguration"> ... <property name="discoverySpi"> <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> <property name="ipFinder"> <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder"/> </property> </bean> </property> </bean>
// Configuring discovery SPI. TcpDiscoverySpi spi = new TcpDiscoverySpi(); // Configuring IP finder. TcpDiscoveryKubernetesIpFinder ipFinder = new TcpDiscoveryKubernetesIpFinder(); spi.setIpFinder(ipFinder); IgniteConfiguration cfg = new IgniteConfiguration(); // Override default discovery SPI. cfg.setDiscoverySpi(spi); // Start Ignite node. Ignition.start(cfg);
ignite-kubernetes as a Maven dependency if you plan to use
TcpDiscoveryKubernetesIpFinder from a source code.
KubernetesIPFinder requires users to configure and deploy a special Kubernetes service that maintains a list of the IP addresses of all the alive Ignite pods (nodes).
Every time a new Ignite pod is started, the IP finder will connect to the service via the Kubernetes API to obtain the list of the existing Ignite pods' addresses. Using these addresses, the new node will be able to discover the rest of the cluster nodes and finally join the Apache Ignite cluster.
The service has to be configured manually and should be started prior to Ignite pods deployment. Below you can see an exemplary configuration for such a service:
apiVersionv1 kindService metadata # Name of Ignite Service used by Kubernetes IP finder for IP addresses lookup. nameignite spec clusterIPNone # custom value. ports port9042 # custom value. selector # Must be equal to one of the labels set in Ignite pods' # deployement configuration. appignite
By default, it's supposed that the service name is
ignite. If you change the name, make sure to adjust
TcpDiscoveryKubernetesIpFinder.setServiceName(String) parameter as well.
Additionally, it's recommended to label Ignite pods in some way and set the label in service's
selector configuration section. For instance, the service started with the configuration above will keep an eye on Kubernetes pods labeled
app: ignite which is the goal.
Connecting to containerized Apache Ignite cluster from outside
TcpDiscoveryKubernetesIpFinder was designed to be used inside of Kubernetes environment meaning that all Apache Ignite nodes, as well as applications that will be interacting with the cluster, are required to be containerized by Kubernetes.
However, if you plan to connect to an Apache Ignite cluster outside of Kubernetes then:
hostNetwork=truein Ignite pod's YAML configuration so that the nodes that will be connecting from outside can establish TCP/IP connections with containerized Ignite nodes (pods).
- Use the same
TcpDiscoveryKubernetesIpFinderfor the nodes that will be outside of Kubernetes environment.
TcpDiscoveryKubernetesIpFinder is designed to be used as is. However, the properties listed below can be adjusted if a fine-grained control is needed:
Sets the name of Kubernetes service for Ignite pods' IP addresses lookup. The name of the service must be equal to the name set in service's Kubernetes configuration.
Sets the namespace the Kubernetes service belongs to.
Sets the host name of the Kubernetes API server.
Specifies the path to the service token file.