Skip to content

Commit 1f65007

Browse files
elena-kolevskaelena-kolevska
committed
diagrams
Signed-off-by: Elena Kolevska <elena@kolevska.com>
1 parent f823d3d commit 1f65007

File tree

7 files changed

+1282
-2
lines changed

7 files changed

+1282
-2
lines changed

‎config.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -182,7 +182,7 @@ def threads_per_client(self) -> int:
182182

183183
@dataclass
184184
class RunnerConfig:
185-
"""Main runner configuration matching lettuce-test-app structure."""
185+
"""Main runner configuration"""
186186
redis: RedisConnectionConfig = field(default_factory=RedisConnectionConfig)
187187
test: TestConfig = field(default_factory=TestConfig)
188188

‎diagrams/README.md‎

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
# Redis Test Application - Complete Architecture Documentation
2+
3+
This directory contains comprehensive diagrams documenting the Redis Test Application's architecture, class structure, code flow, and data flow.
4+
5+
## 📋 Diagram Overview
6+
7+
### 1. [Class Diagram](./class-diagram.md)
8+
**Purpose**: Shows all classes, their properties, methods, and relationships
9+
**Key Components**:
10+
- **Configuration Classes**: `RunnerConfig`, `RedisConnectionConfig`, `TestConfig`, `WorkloadConfig`
11+
- **Core Engine Classes**: `TestRunner`, `RedisClientManager`, `RedisClientPool`
12+
- **Workload Classes**: `BaseWorkload` and 8 specialized implementations
13+
- **Observability Classes**: `MetricsCollector`, `OperationMetrics`, `RedisTestLogger`
14+
- **CLI Classes**: Command-line interface components
15+
16+
**Relationships**:
17+
- Composition relationships between config classes
18+
- Inheritance hierarchy for workload classes
19+
- Dependency injection patterns for metrics and logging
20+
21+
### 2. [Code Flow Diagram](./code-flow-diagram.md)
22+
**Purpose**: Illustrates the execution flow from application startup to operation completion
23+
**Key Flows**:
24+
- **Application Startup**: CLI → Configuration → TestRunner initialization
25+
- **Thread Architecture**: Main thread spawning worker threads and stats reporter
26+
- **Redis Operations**: Operation selection → execution → metrics recording
27+
- **Metrics Collection**: Data aggregation → OpenTelemetry/Prometheus export
28+
29+
### 3. [Data Flow Diagram](./data-flow-diagram.md)
30+
**Purpose**: Shows how data moves through the system components
31+
**Key Data Flows**:
32+
- **Configuration Data**: Environment variables → CLI args → merged config
33+
- **Redis Connection Data**: Config → connection setup → pool management
34+
- **Operation Data**: Key/value generation → Redis commands → response processing
35+
- **Metrics Data**: Operation results → aggregation → export to observability stack
36+
37+
### 4. [System Architecture Diagram](./system-architecture-diagram.md)
38+
**Purpose**: Provides high-level system architecture and deployment views
39+
**Key Architectures**:
40+
- **Overall System**: Layered architecture with clear separation of concerns
41+
- **Component Interaction**: Runtime relationships between major components
42+
- **Deployment Architecture**: Docker containerization and service orchestration
43+
- **Error Handling**: Resilience patterns and failure recovery mechanisms
44+
45+
## 🏗️ Architecture Highlights
46+
47+
### Multi-Layered Design
48+
```
49+
┌─────────────────────────────────────┐
50+
│ CLI Layer │
51+
├─────────────────────────────────────┤
52+
│ Configuration Layer │
53+
├─────────────────────────────────────┤
54+
│ Core Engine │
55+
├─────────────────────────────────────┤
56+
│ Redis Connectivity │
57+
├─────────────────────────────────────┤
58+
│ Workload Engine │
59+
├─────────────────────────────────────┤
60+
│ Observability Layer │
61+
└─────────────────────────────────────┘
62+
```
63+
64+
### Key Design Patterns
65+
66+
1. **Factory Pattern**: `WorkloadFactory` creates appropriate workload instances
67+
2. **Pool Pattern**: `RedisClientPool` manages connection pooling
68+
3. **Observer Pattern**: Metrics collection observes operation results
69+
4. **Strategy Pattern**: Different workload implementations for various test scenarios
70+
5. **Singleton Pattern**: Global metrics collector and logger instances
71+
72+
### Thread Architecture
73+
74+
```
75+
TestRunner (Main)
76+
├── Stats Reporter Thread (1x)
77+
├── Client Pool 1
78+
│ ├── Worker Thread 1-1
79+
│ ├── Worker Thread 1-2
80+
│ └── Worker Thread 1-N
81+
├── Client Pool 2
82+
│ ├── Worker Thread 2-1
83+
│ ├── Worker Thread 2-2
84+
│ └── Worker Thread 2-N
85+
└── Client Pool N (client_instances)
86+
├── Worker Thread N-1
87+
├── Worker Thread N-2
88+
└── Worker Thread N-N
89+
```
90+
91+
**Total Threads**: `client_instances × threads_per_client + 1 (stats reporter)`
92+
93+
### Configuration Hierarchy
94+
95+
```
96+
RunnerConfig
97+
├── RedisConnectionConfig
98+
│ ├── Connection parameters (host, port, auth)
99+
│ ├── SSL/TLS configuration
100+
│ ├── Cluster mode settings
101+
│ └── Connection pool settings
102+
├── TestConfig
103+
│ ├── Threading configuration
104+
│ ├── Duration and rate limiting
105+
│ └── WorkloadConfig
106+
│ ├── Workload type selection
107+
│ ├── Operation parameters
108+
│ └── Workload-specific options
109+
├── Metrics Configuration
110+
│ ├── OpenTelemetry settings
111+
│ ├── Prometheus configuration
112+
│ └── Export intervals
113+
└── Logging Configuration
114+
├── Log levels
115+
├── File output
116+
└── Console formatting
117+
```
118+
119+
### Workload Types and Operations
120+
121+
| Workload Type | Primary Operations | Use Case |
122+
|---------------|-------------------|----------|
123+
| `basic_rw` | SET, GET, DEL, INCR | General Redis testing |
124+
| `high_throughput` | Optimized operations | Performance testing |
125+
| `list_operations` | LPUSH, RPOP, LLEN | List data structure testing |
126+
| `set_operations` | SADD, SREM, SCARD | Set data structure testing |
127+
| `sorted_set_operations` | ZADD, ZREM, ZCARD | Sorted set testing |
128+
| `hash_operations` | HSET, HGET, HDEL | Hash data structure testing |
129+
| `pipeline` | Batched operations | Pipeline performance testing |
130+
| `pub_sub` | PUBLISH, SUBSCRIBE | Pub/Sub functionality testing |
131+
132+
### Metrics Collection Strategy
133+
134+
The application uses a **push-based metrics collection** approach:
135+
136+
1. **Operation Level**: Each Redis operation records metrics
137+
2. **Thread Level**: Worker threads aggregate local metrics
138+
3. **Application Level**: `MetricsCollector` centralizes all metrics
139+
4. **Export Level**: OpenTelemetry exports to OTLP Collector
140+
5. **Visualization Level**: Grafana displays metrics from Prometheus endpoint
141+
142+
### Error Handling and Resilience
143+
144+
- **Connection Resilience**: Automatic reconnection with exponential backoff
145+
- **Operation Retry**: Configurable retry logic for failed operations
146+
- **Pool Management**: Dynamic client pool sizing and health monitoring
147+
- **Graceful Degradation**: Continues operation with reduced capacity during failures
148+
- **Comprehensive Logging**: Detailed error tracking and debugging information
149+
150+
## 🚀 Getting Started with the Architecture
151+
152+
1. **Start with Class Diagram**: Understand the core classes and their relationships
153+
2. **Follow Code Flow**: Trace execution from startup through operation completion
154+
3. **Understand Data Flow**: See how configuration and operation data moves through the system
155+
4. **Review System Architecture**: Get the big picture of component interactions
156+
5. **Explore Deployment**: Understand how the system runs in containerized environments
157+
158+
## 📊 Observability Integration
159+
160+
The application integrates with modern observability stacks:
161+
162+
- **OpenTelemetry**: Standards-based metrics and tracing
163+
- **Prometheus**: Time-series metrics storage and alerting
164+
- **Grafana**: Rich visualization and dashboarding
165+
- **Jaeger**: Distributed tracing and performance analysis
166+
167+
This comprehensive architecture enables high-performance Redis testing with full observability and operational insights.

0 commit comments

Comments
 (0)