MongoDB switch-over from VM to AKS

MongoDB, a popular open-source NoSQL database management system, distinguishes itself from traditional SQL databases by adopting a flexible approach to data storage. In SQL, databases are structured with tables and a rigid schema, while MongoDB utilizes collections similar to tables. However, MongoDB introduces a dynamic aspect with its document-oriented model, allowing nested data storage. This capability empowers users to establish intricate relationships within a single document, enhancing data retrieval and manipulation efficiency. Unlike SQL’s vertical scaling, MongoDB offers horizontal scaling, making it well-suited for dynamic, scalable, and evolving datasets. The emphasis on performance over strict consistency caters to scenarios where adaptability and speed are paramount, making MongoDB an ideal choice for managing modern, diverse data types.

MongoDB migration to Kubernetes entails several crucial steps to integrate the database seamlessly into the container orchestration environment. First and foremost, the MongoDB instance must be containerized, allowing it to run within a Kubernetes cluster. Deployment is facilitated through Kubernetes manifests, specifying the necessary configurations and resource requirements. Persistent storage solutions are configured to ensure data retention and durability across container restarts. Security considerations are addressed by managing secrets and configurations through tools like Vault, safeguarding sensitive information.

Service discovery mechanisms are also implemented to enable effective communication within the Kubernetes cluster. Monitoring and logging strategies are crucial for the migration process to optimize performance and maintain operational visibility. Lastly, scaling strategies are devised to accommodate varying workloads and ensure the efficient utilization of resources, contributing to the overall success of MongoDB integration within a Kubernetes environment.

• Scalability Constraints:

The fixed nature of VMs restricts easy scalability. Increasing resources often requires downtime or complex scaling procedures. 

Impact: Inability to swiftly adjust resources in response to varying workloads or business demands, leading to potential performance bottlenecks during peak usage.

• Resource Management Difficulties:

Allocating and managing resources across VMs can be complex. VMs might need to utilize resources more optimally.

Impact: 

Inefficient resource utilization leads to underutilized or overburdened VMs, impacting performance and cost-efficiency.

• Deployment Inflexibility:

Deploying updates or changes across VMs can be challenging and time-consuming.

Impact: 

Slow deployment cycles, increased chances of errors during updates, and difficulties maintaining consistency across the environment.

• Enhanced Scalability:

Kubernetes allows for seamless horizontal scaling, enabling resources to be easily added or removed based on demand.

Impact: 

MongoDB instances can dynamically adapt to varying workloads, ensuring optimal performance during peak usage without downtime.

• Improved Resource Utilization:

Kubernetes efficiently manages resources by allocating them precisely where needed, avoiding underutilization or overprovisioning.

Impact: 

Optimal resource usage minimizes costs and performance, enhancing overall database efficiency.

• Flexible Deployment and Updates:

Kubernetes facilitates automated deployment and updates, reducing deployment cycles and complexities.

Impact: 

Faster deployment processes enable quick iterations and updates, enhancing system agility and change responsiveness.

5.1 Review of Current Infrastructure

The current infrastructure consists of a MongoDB replica set with 3 replicas deployed across 3 VMS.

• MongoDB version – 5.0
• DB size – 50GB
• Network – All VMs deployed under the same subnet in VPC alpha.

• Networking Setup: Configure VPC peering to ensure connectivity between the VM and Kubernetes clusters. Validate and ensure seamless interconnectivity among all MongoDB pods and associated VMs to guarantee robust communication across the entire environment.
• Hostname Entries: Establish and document hostname entries for all virtual machines (VMs) within each MongoDB pod. 
• Replication key: Integrate and synchronize the replication key used in MongoDB VM replication with Vault
• Backup Live Cluster: Before commencing migration, ensure a complete backup of the existing live MongoDB cluster is taken for data security and recovery purposes.
• External Traffic Policy: Modify the external traffic policy to prioritize local traffic within the Kubernetes environment.

5.3.1 Pod Configuration

Set up MongoDB instances with hostnames in the new Kubernetes cluster and expose services as NodePort for external connectivity.

5.3.2 Cluster Integration

Add the new MongoDB pods as slaves to the existing cluster. The replica-set can trigger a leader election in response to adding replica events; hence, we should configure the new members to be excluded from the replica-set leader election during the adding and synchronization stage by lowering the vote and priority for all new instances.

Example

Check the rs.status and wait for db sync to be completed.

Repeat this to add the other 2 instances.

5.3.3 Changing priority

5.3.4 Remove old secondaries

5.3.5 Switch primary

5.3.6 Remove old primary

5.3.7 Change hostnames

7. Challenges faced during Migration

• After the primary switch, the application had an error ‘NotWritablePrimary’.
• Replicated the issue locally and revised steps.

1. Scale down app-b to 0
2. Remove Secondary DB VMs from the cluster.

3. Switch primary to k8s mongo-pod.

4. Remove Primary DB VMs from the cluster.

5. Update K8s Mongo secondary pods hostaddress from  NodeIP: NodePort to Podname.svcname

6. Check the health of the Mongo replica set by capturing rs.conf and rs.status
7. Deploy ‘app-g’ with configuration pointing to the MongoDB pods by updating the ‘appz.yaml’ file.
8. Perform application testing.
9. Update the ‘app-svc’ service to reference ‘app-g’.
10. Execute a basic performance test using the live domain.

Conclusion

Migrating MongoDB to a Kubernetes cluster represents a strategic leap toward improved scalability and operational efficiency. This integration streamlines deployment processes and optimizes resource utilization, which is especially beneficial for contemporary, data-intensive applications. By harnessing the dynamic orchestration capabilities of Kubernetes, MongoDB can adapt seamlessly to varying workloads, ensuring a more agile and responsive infrastructure. This synergy between MongoDB’s database strengths and Kubernetes’ flexible orchestration paves the way for a modern, efficient, and scalable solution for managing complex datasets.