High-Performance IoT Monitoring With Redis-Backed In-Memory Storage and Containerized CI/CD Pipeline

This study presents the design, development, and deployment of a high-performance Internet of Things (IoT) monitoring system featuring an integrated Redis-based in-memory processing architecture. The system collects real-time environmental data—including temperature, humidity, and pressure—from field-deployed TI CC1352R SensorTag devices operating within an LR-WPAN network. These measurements are forwarded to a TI CC1352R LaunchPad border router node, which employs an ESP32 Wi-Fi SoC interface to transmit the sensed data to cloud-based web services via the global Internet. The web service dynamically caches and stores incoming traffic according to database write speed and network intensity, while the Redis caching layer significantly enhances data ingestion and monitoring response times by reducing database load. The backend follows modular and scalable CI/CD pipeline principles, automating the testing, building, and deployment phases. Except for the firmware running on sensor nodes, every component of the system is triggered by version-controlled merges, ensuring automated testing and re-deployment. One of the main challenges in complex IoT systems—especially those requiring continuous weekly feature updates (sprint-based development)—is constructing a fully integrated, end-to-end system that remains stable and extensible. Experimental findings demonstrate rapid recovery from fault conditions (with total re-deployment times below 10 minutes), efficient real-time data visualization, and improved system resilience. The proposed architecture also provides practical guidance for researchers focusing on IoT systems that traditionally rely on simulations by offering a fully operational, end-to-end technological solution. By caching large volumes of sensor data before disk storage, the system overcomes one of the major scalability bottlenecks in growing IoT networks. On a 12 GB RAM server, the architecture successfully stores data from approximately 530,000 sensor messages using a 10-second caching interval, effectively absorbing sudden data bursts through Redis buffering. Consequently, the study establishes a robust foundation for future extensions involving machine learning integration and large-scale sensor deployments.