Skip to content

Commit 5d08e94

Browse files
committed
feat(vcr): add experimental VCR test system for LLM/embedding recording
Implements a VCR (Video Cassette Recorder) test system that enables recording and replaying LLM and embedding API calls for deterministic, fast, and cost-effective testing. Core components: - VCRMode: 6 modes (PLAYBACK, RECORD, RECORD_NEW, RECORD_FAILED, PLAYBACK_OR_RECORD, OFF) with smart mode selection - VCRExtension: JUnit 5 extension with full lifecycle callbacks - VCRContext: Redis container management with AOF/RDB persistence - VCRRegistry: Test recording status tracking with smart mode logic - @VCRTest, @VCRRecord, @VCRDisabled: Annotations for test configuration Key features: - Redis-based cassette storage with persistence to src/test/resources/vcr-data - Testcontainers integration for isolated Redis instances - Call counter management for deterministic key generation - Statistics tracking (cache hits, misses, API calls) - Configurable data directory and Redis image Also includes: - Comprehensive documentation in docs/experimental section - README section with quick start example - Design documents for VCR system and EmbeddingsCache enhancement All 39 VCR tests passing. LLM/embedding interceptor implementation pending.
1 parent de520da commit 5d08e94

File tree

16 files changed

+2843
-0
lines changed

16 files changed

+2843
-0
lines changed

README.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -376,6 +376,34 @@ System.out.println(match.getDistance()); // Output: 0.273891836405
376376
377377
> Learn more about [semantic routing](https://redis.github.io/redis-vl-java/redisvl/current/semantic-router.html).
378378
379+
## 🧪 Experimental: VCR Test System
380+
381+
RedisVL includes an experimental VCR (Video Cassette Recorder) test system for recording and replaying LLM/embedding API calls. This enables:
382+
383+
- **Deterministic tests** - Replay recorded responses for consistent results
384+
- **Cost reduction** - Avoid repeated API calls during test runs
385+
- **Speed improvement** - Local Redis playback is faster than API calls
386+
- **Offline testing** - Run tests without network access
387+
388+
```java
389+
import com.redis.vl.test.vcr.VCRTest;
390+
import com.redis.vl.test.vcr.VCRMode;
391+
392+
@VCRTest(mode = VCRMode.PLAYBACK_OR_RECORD)
393+
public class MyLLMTest {
394+
395+
@Test
396+
void testLLMResponse() {
397+
// First run: Records API response to Redis
398+
// Subsequent runs: Replays from Redis cassette
399+
String response = myLLMService.generate("What is Redis?");
400+
assertNotNull(response);
401+
}
402+
}
403+
```
404+
405+
> Learn more about [VCR testing](https://redis.github.io/redis-vl-java/redisvl/current/vcr-testing.html).
406+
379407
## 🚀 Why RedisVL?
380408
381409
In the age of GenAI, **vector databases** and **LLMs** are transforming information retrieval systems. With emerging and popular frameworks like [LangChain4J](https://github.com/langchain4j/langchain4j) and [Spring AI](https://spring.io/projects/spring-ai), innovation is rapid. Yet, many organizations face the challenge of delivering AI solutions **quickly** and at **scale**.

core/build.gradle.kts

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,16 @@ dependencies {
6565
// Cohere Java SDK for reranking
6666
compileOnly("com.cohere:cohere-java:1.8.1")
6767

68+
// VCR Test Utilities (optional - users include what they need for testing)
69+
// JUnit 5 for extension development
70+
compileOnly("org.junit.jupiter:junit-jupiter-api:5.10.2")
71+
// Testcontainers for Redis persistence
72+
compileOnly("org.testcontainers:testcontainers:1.19.7")
73+
compileOnly("org.testcontainers:junit-jupiter:1.19.7")
74+
// ByteBuddy for method interception (future LLM interceptor)
75+
compileOnly("net.bytebuddy:byte-buddy:1.14.12")
76+
compileOnly("net.bytebuddy:byte-buddy-agent:1.14.12")
77+
6878
// Test dependencies for LangChain4J (include in tests to verify integration)
6979
testImplementation("dev.langchain4j:langchain4j:0.36.2")
7080
testImplementation("dev.langchain4j:langchain4j-open-ai:0.36.2")
@@ -79,6 +89,12 @@ dependencies {
7989
// Additional test dependencies
8090
testImplementation("com.squareup.okhttp3:mockwebserver:4.12.0")
8191
testImplementation("org.mockito:mockito-core:5.11.0")
92+
93+
// VCR test dependencies (to test VCR functionality)
94+
testImplementation("org.testcontainers:testcontainers:1.19.7")
95+
testImplementation("org.testcontainers:junit-jupiter:1.19.7")
96+
testImplementation("net.bytebuddy:byte-buddy:1.14.12")
97+
testImplementation("net.bytebuddy:byte-buddy-agent:1.14.12")
8298
}
8399

84100
// Configure test execution

0 commit comments

Comments
 (0)