Skip to content

Commit e3122c7

Browse files
author
Paweł Kędzia
committed
Merge branch 'features/refactor'
2 parents dcd3fd4 + a81dd6c commit e3122c7

33 files changed

+436
-381
lines changed

CHANGELOG.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,4 +16,4 @@
1616
| 0.3.0 | Anonymization available with three strategies: `fast_masker`, `genai`, `prov_masker`. |
1717
| 0.3.1 | Refactoring `lb.strategies` to be more flexible modular. Introduced `MaskerPipeline` and `GuardrailPipeline` both configured via env. Removed genai-based masking endpoint. |
1818
| 0.4.0 | The main repository is divided into dedicated ones: plugins, services, web — separate repositories. Clean up the whole repository. Examples of integration with llamaindex, langchain, openai, litellm and haystack. |
19-
| 0.4.1 | Audit log is stored using GPG. Add bash script (`scripts/gen_and_export_gpg.sh` to prepare GPG keys and simple `scripts/decrypt_auditor_logs.sh` to decrypt encrypted audit logs. |
19+
| 0.4.1 | Audit log is stored using GPG. Add bash script (`scripts/gen_and_export_gpg.sh` to prepare GPG keys and simple `scripts/decrypt_auditor_logs.sh` to decrypt encrypted audit logs. Moved core functionality from `base` to module `core` module. |

llm_router_api/LB_STRATEGIES.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ and reliable routing of requests. The available strategies are:
1111
been used for a specific model. It selects the provider that has been used the least.
1212
* **When to use:** Ideal for scenarios where all providers are considered equal
1313
in terms of capacity and performance. It provides a simple and effective way to balance the load.
14-
* **Implementation:** Implemented in `llm_router_api.base.lb.balanced.LoadBalancedStrategy`.
14+
* **Implementation:** Implemented in `llm_router_api.core.lb.balanced.LoadBalancedStrategy`.
1515

1616
### 2. `weighted`
1717

@@ -20,7 +20,7 @@ and reliable routing of requests. The available strategies are:
2020
ensuring that over time, the request distribution closely matches the configured weights.
2121
* **When to use:** Useful when you have providers with different capacities or performance
2222
characteristics, and you want to prioritize certain providers without needing dynamic adjustments.
23-
* **Implementation:** Implemented in `llm_router_api.base.lb.weighted.WeightedStrategy`.
23+
* **Implementation:** Implemented in `llm_router_api.core.lb.weighted.WeightedStrategy`.
2424

2525
### 3. `dynamic_weighted` (beta)
2626

@@ -31,7 +31,7 @@ and reliable routing of requests. The available strategies are:
3131
* **When to use:** Recommended for dynamic environments where provider performance
3232
can fluctuate. It offers more sophisticated load balancing by considering both
3333
configured weights and real-time performance metrics (latency).
34-
* **Implementation:** Implemented in `llm_router_api.base.lb.weighted.DynamicWeightedStrategy`.
34+
* **Implementation:** Implemented in `llm_router_api.core.lb.weighted.DynamicWeightedStrategy`.
3535

3636
### 4. `first_available`
3737

@@ -42,7 +42,7 @@ and reliable routing of requests. The available strategies are:
4242
possible response and want to ensure that a request is immediately handled by any available
4343
provider, without complex load distribution logic. It guarantees that a provider,
4444
once taken, is exclusive until released.
45-
* **Implementation:** Implemented in `llm_router_api.base.lb.first_available.FirstAvailableStrategy`.
45+
* **Implementation:** Implemented in `llm_router_api.core.lb.first_available.FirstAvailableStrategy`.
4646

4747
**When using the** `first_available` load balancing strategy, a **Redis server is required**
4848
for coordinating provider availability across multiple workers.
@@ -100,7 +100,7 @@ To use a different strategy (e.g., round‑robin, random weighted, latency‑bas
100100
implement `ChooseProviderStrategyI` and pass the instance to `ProviderChooser`:
101101

102102
``` python
103-
from llm_router_api.base.lb.chooser import ProviderChooser
103+
from llm_router_api.core.lb.chooser import ProviderChooser
104104
from my_strategies import RoundRobinStrategy
105105

106106
chooser = ProviderChooser(strategy=RoundRobinStrategy())

llm_router_api/base/lb/strategies/first_available_optim.py

Lines changed: 0 additions & 300 deletions
This file was deleted.

llm_router_api/core/engine.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@
3333
USE_PROMETHEUS,
3434
SERVER_BALANCE_STRATEGY,
3535
)
36-
from llm_router_api.base.lb.provider_strategy_facade import ProviderStrategyFacade
36+
from llm_router_api.core.lb.provider_strategy_facade import ProviderStrategyFacade
3737

3838
if USE_PROMETHEUS:
3939
from llm_router_api.core.metrics import PrometheusMetrics

0 commit comments

Comments
 (0)