Zookeeper: Coordination in Distributed Systems

Zookeeper is a distributed coordination service that simplifies the management of distributed systems. Let’s explore its role and how to integrate it with Spring microservices.

What is Zookeeper?

Zookeeper is a centralized service for maintaining configuration, naming, synchronization, and group services in distributed systems.

Client → Zookeeper → Distributed System

Key Features

1. Configuration Management:

   • Store and retrieve configuration data.

2. Service Discovery:

   • Locate services dynamically.

3. Leader Election:

   • Elect a leader among distributed nodes.

4. Distributed Locks:

   • Ensure mutual exclusion in distributed systems.

5. Event Notifications:

   • Notify clients of changes in data.

Common Patterns

1. Configuration Management

Store configuration data in Zookeeper and retrieve it dynamically.

Zookeeper Node:
/config/app/db → jdbc:mysql://localhost:3306/mydb

2. Service Discovery

Register services with Zookeeper and let clients discover them.

Zookeeper Node:
/services/order-service → 192.168.1.10:8080

3. Leader Election

Elect a leader among distributed nodes for coordination.

Zookeeper Node:
/election → Node 1 (Leader)

Zookeeper in Spring Microservices

Dependencies

<dependency>
    <groupId>org.apache.curator</groupId>
    <artifactId>curator-framework</artifactId>
</dependency>
<dependency>
    <groupId>org.apache.curator</groupId>
    <artifactId>curator-recipes</artifactId>
</dependency>

Configuration

zookeeper:
  connect-string: localhost:2181
  session-timeout: 5000
  connection-timeout: 3000

Service Discovery Example

Registering a Service

@Service
public class ServiceRegistrar {
    private final CuratorFramework client;

    public ServiceRegistrar(CuratorFramework client) {
        this.client = client;
    }

    public void registerService(String serviceName, String address) throws Exception {
        String path = "/services/" + serviceName;
        client.create().creatingParentsIfNeeded().forPath(path, address.getBytes());
    }
}

Discovering a Service

@Service
public class ServiceDiscovery {
    private final CuratorFramework client;

    public ServiceDiscovery(CuratorFramework client) {
        this.client = client;
    }

    public String discoverService(String serviceName) throws Exception {
        String path = "/services/" + serviceName;
        return new String(client.getData().forPath(path));
    }
}

Leader Election Example

@Service
public class LeaderElection {
    private final LeaderSelector leaderSelector;

    public LeaderElection(CuratorFramework client) {
        this.leaderSelector = new LeaderSelector(client, "/election", new LeaderSelectorListenerAdapter() {
            @Override
            public void takeLeadership(CuratorFramework client) throws Exception {
                System.out.println("I am the leader now!");
                Thread.sleep(5000); // Simulate leadership work
            }
        });
        this.leaderSelector.autoRequeue();
    }

    public void start() {
        leaderSelector.start();
    }
}

Challenges

1. Single Point of Failure

• Mitigation: Use Zookeeper in a cluster mode.

2. Operational Overhead

• Mitigation: Use managed services like AWS Zookeeper or Apache Kafka’s KRaft mode.

3. Latency

• Mitigation: Minimize frequent writes to Zookeeper.

When to Use Zookeeper

1. Distributed Coordination:

   • Leader election, distributed locks, etc.

2. Service Discovery:

   • Dynamically locate services in a distributed system.

3. Configuration Management:

   • Centralized configuration for distributed nodes.

When Not to Use Zookeeper

1. Simple Systems:

   • Avoid unnecessary complexity for small applications.

2. High Write Workloads:

   • Zookeeper is optimized for reads, not writes.

Bottom Line

1. Zookeeper simplifies coordination in distributed systems.
2. Use it for service discovery, leader election, and configuration management.
3. Avoid overloading Zookeeper with frequent writes or unnecessary complexity.

Remember: Zookeeper is a powerful tool for distributed systems, but it comes with operational overhead. Use it wisely.