Skip to content

Commit 9c990fd

Browse files
jerrinotpuzpuzpuz
andauthored
enterprise quick start to use ILP@HTTP instead of ILP@TCP (#205)
Co-authored-by: Andrei Pechkurov <37772591+puzpuzpuz@users.noreply.github.com>
1 parent 3901ad3 commit 9c990fd

File tree

1 file changed

+93
-93
lines changed

1 file changed

+93
-93
lines changed

documentation/guides/enterprise-quick-start.md

Lines changed: 93 additions & 93 deletions
Original file line numberDiff line numberDiff line change
@@ -184,152 +184,152 @@ List the full list of applied permissions with `all_permissions()`.
184184

185185
## 4. Ingest data, InfluxDB Line Protocol
186186

187-
Perform data ingestion through a service account or an interactive user. Service
188-
accounts are recommended over users, as they apply a cleaner set of access
189-
permissions, and are less likely to be affected by day-to-day user management
190-
operations.
187+
The recommended method for high-throughput ingestion is InfluxDB Line Protocol (ILP) over HTTP.
191188

192-
First, set up the service account. Then, use it to create a token which is
193-
associated with the service account. This token is then provided to your
194-
InfluxDB client to form a secure, access-controlled connection.
189+
We recommend using a service account for programmatic ingestion. Service accounts apply a cleaner set of access
190+
permissions and are less likely to be affected by day-to-day user management.
195191

196-
A service account is "an account for a service". This is in contrast to an
197-
account for a user. When service accounts are created, we assume that they
198-
belong to an organization and not an individual. Your sensors, apps or cars may
199-
use service accounts. Sam, the plucky analyst, may have an interactive user.
192+
The process is:
200193

201-
The recommended ingestion method is via the InfluxDB Line Protocol (ILP).
194+
1. Create a service account and grant it permissions.
195+
2. Generate a **REST token** for the service account.
196+
3. Use this token in your client's connection string.
202197

203-
To setup a service account:
198+
### Step 1: Create the Service Account
199+
200+
First, run the following SQL in the web console. This creates a service account named `ingest_http` and grants it the
201+
necessary permissions to use HTTP endpoints and manage data.
204202

205203
```questdb-sql title="Web Console - Setup a service account"
206-
CREATE SERVICE ACCOUNT ingest;
207-
GRANT ilp, create table TO ingest;
208-
GRANT add column, insert ON all tables TO ingest;
209-
-- OR
210-
GRANT add column, insert ON table1, table2 TO ingest;
204+
CREATE SERVICE ACCOUNT ingest_ilp;
205+
-- Grant permission to create tables and use HTTP endpoints
206+
GRANT HTTP, CREATE TABLE TO ingest_ilp;
207+
-- Grant permission to add columns and insert data
208+
GRANT ADD COLUMN, INSERT ON ALL TABLES TO ingest_ilp;
209+
210+
-- OR, for more granular control:
211+
-- GRANT ADD COLUMN, INSERT ON table1, table2 TO ingest_ilp;
211212
```
212213

213-
This creates a service account called `ingest`, which:
214-
215-
- Can create a table
216-
- Add table columns
217-
- Insert to all tables OR insert to specific tables
214+
### Step 2: Generate an Authentication Token
218215

219-
The account exists, and that means that a client of some kind can connect to
220-
QuestDB via that account. Next, create a token to create a secure link between
221-
the client and the account:
216+
Next, generate a REST API token for the service account. This token acts as a password, so you must store it securely.
222217

223-
```questdb-sql title="Web Console - Generate a token for ingest client"
224-
ALTER SERVICE ACCOUNT ingest CREATE TOKEN TYPE JWK;
218+
```questdb-sql title="Web Console - Generate a token for the ingest client"
219+
ALTER SERVICE ACCOUNT ingest_ilp CREATE TOKEN TYPE REST WITH TTL '3000d' REFRESH;
225220
```
226221

227-
This creates a token comprised of three parts:
228-
229-
- public_key_x
230-
- public_key_y
231-
- private_key
232-
233-
| name | public_key_x | public_key_y | private_key |
234-
| ------ | ------------------------------------------ | ------------------------------------------ | ------------------------------------------- |
235-
| ingest | gxVbx90=MtYMmIEek2L5jFa5e9qTIvxI2TKSfI3GVE | kEUZjIfU9=S6w6uR=j130v003YgB3NBpYcVswvvacs | kom7j38LG44HcPfO92oZ4558e6KoeTHn6H5rA8vK3PQ |
236-
237-
Now, this private key is then added to the client.
222+
This command returns a token. **Copy it immediately**, as it's shown only once.
238223

239-
This provides authenticated access to QuestDB for the "ingest" user.
224+
| name | token | expires_at | refresh |
225+
|------------|------------------------------------------------|-----------------------------|---------|
226+
| ingest_ilp | qt1KAsf1U9YbUVAX1H2IahXEE3-4qBcK-zx_jsZUzV9bLY | 2033-09-19T15:32:51.628453Z | true |
240227

241-
For example, if you are leveraging Java and our recommended InfluxDB Line Protocol over HTTP client:
228+
### Step 3: Use the Token in Your Client
242229

243-
```java
244-
Java client example:
230+
You can now use this token to authenticate your application. The following Java example shows how to use the client
231+
library by configuring it from a connection string. This is the recommended approach.
245232

233+
```java title="Java - Ingesting data via ILP"
246234
import io.questdb.client.Sender;
247235
import java.time.temporal.ChronoUnit;
248236

249-
public class ILPMain {
237+
public class Ingest {
250238
public static void main(String[] args) {
251-
try (Sender sender = Sender.builder(Sender.Transport.HTTP)
252-
.address("localhost:9000")
253-
.enableTls()
254-
.enableAuth("ingest")
255-
.authToken("kom7j38LG44HcPfO92oZ4558e6KoeTHn6H5rA8vK3PQ")
256-
.build()) {
257-
239+
try (Sender sender = Sender.fromConfig("https::addr=localhost:9000;token=qt1KAsf1U9YbUVAX1H2IahXEE3-4qBcK-zx_jsZUzV9bLY;")) {
258240
sender.table("ilptest");
259241
sender.symbol("sym1", "symval1")
260-
.doubleColumn("double1", 100.0)
261-
.at(System.currentTimeMillis(), ChronoUnit.MILLIS);
242+
.doubleColumn("double1", 100.0)
243+
.at(System.currentTimeMillis(), ChronoUnit.MILLIS);
262244
}
263245
}
264246
}
265247
```
266248

267-
Please note that the private key is not stored in the database.
268-
269-
There is **no way to get your private key back** at a later time!
270-
271-
Once generated, safely store it.
249+
:::note A Note on TLS
250+
The `https::` prefix in the connection string tells the client to connect using TLS. By default, the client will verify
251+
the server's certificate. For local testing with self-signed certificates, you can disable this validation by adding
252+
`tls.verify=insecure;` to the configuration string. **This is not recommended for production.**
253+
:::
272254

273255
Connecting a client to ILP is a common path.
274256

275257
However, you may use something like [Kafka](/docs/third-party-tools/kafka).
276258

277259
## 5. Ingest data, Kafka Connect (optional)
278260

279-
_If you're not applying Kafka, skip to step 6._
261+
_If you're not using Kafka, you can skip to section 6._
280262

281-
The
282-
[Kafka Connect](https://docs.confluent.io/platform/current/connect/index.html)
283-
connector can be thought of as a specialized ILP client.
263+
The official **QuestDB Kafka Connect sink** forwards messages from Kafka topics directly to your database using ILP protocol.
264+
The setup process is straightforward:
284265

285-
Thus the steps are similar to ILP ingestion:
266+
1. Create a dedicated service account in QuestDB.
267+
2. Generate an authentication token for the account.
268+
3. Configure the Kafka sink connector with your QuestDB address and the token.
286269

287-
1. Create a Kafka service account and assign permissions
288-
1. Configure the connector to use the service account
270+
### **Step 1: Create the Service Account**
289271

290-
```questdb-sql title="Web Console - Create a service account and grant required permissions"
272+
In the QuestDB web console, create a service account named `kafka` and grant it the permissions required to connect and
273+
write data.
274+
275+
```questdb-sql title="Web Console - Create a Kafka service account"
291276
CREATE SERVICE ACCOUNT kafka;
292-
GRANT ilp, create table TO kafka;
293-
GRANT add column, insert ON all tables TO kafka;
294-
OR
295-
GRANT add column, insert ON table1, table2 to kafka;
277+
278+
-- Grant permissions to use HTTP, create tables, add new columns and insert data
279+
GRANT HTTP, CREATE TABLE TO kafka;
280+
GRANT ADD COLUMN, INSERT ON ALL TABLES TO kafka;
281+
282+
-- OR, for more granular control:
283+
-- GRANT ADD COLUMN, INSERT ON table1, table2 TO ingest_ilp;
296284
```
297285

286+
### **Step 2: Generate an Authentication Token**
287+
288+
Next, generate a REST API token for the `kafka` service account. This token is a secret credential and should be treated like a
289+
password.
290+
298291
```questdb-sql title="Web Console - Generate a token for the service account"
299-
ALTER SERVICE ACCOUNT kafka CREATE TOKEN TYPE JWK;
292+
-- Creates a token that is valid for 1 year (365 days)
293+
ALTER SERVICE ACCOUNT kafka CREATE TOKEN TYPE REST WITH TTL '365d';
300294
```
301295

302-
The token SQL returns a multi-part token, as before:
296+
The command returns a token. **Copy it immediately**, as it will not be shown again.
303297

304-
| name | public_key_x | public_key_y | private_key |
305-
| ----- | ----------------------------------------- | ------------------------------------------- | --------------------------------------------- |
306-
| kafka | uE3MxG5_PMTor0V40LIBNTaUv-xh0dMPRN83nsQUI | 73wq8nx02Pj6W6yt53VxFT9K-TnopdM0s0UeeBxTgb0 | tDNC3DJ_L_QYIZRu_L4S4YTZYZXbjr7JX_bxFYdHhhhQU |
307-
308-
Remember, the private key cannot be retrieved.
298+
| name | token | expires\_at |
299+
|-------|--------------------------------------------------|-------------------------------|
300+
| kafka | `qt1KAsf1U9YbUVAX1H2IahXEE3-4qBcK-zx_jsZUzV9bLY` | `2026-07-03T18:05:00.000000Z` |
309301

310302
Save the private key in a secure location!
311303

312-
Next, configure the username, token and TLS settings inside of Kafka:
304+
### **Step 3: Configure the Kafka Connect Sink**
305+
306+
Create a configuration file for the QuestDB sink connector. In the `client.conf.string` property, provide your QuestDB
307+
server address and paste the token you just generated.
313308

314-
```bash title="Kafka Connect - Configuring the QuestDB Kafka connector"
315-
username=kafka
316-
token=[the private key saved in the previous step]
317-
tls=true
309+
```properties title="questdb-sink.properties"
310+
# --- Connector Identity ---
311+
name=QuestDBSinkConnector
312+
connector.class=io.questdb.kafka.QuestDBSinkConnector
313+
tasks.max=1
318314

319-
# If QuestDB server does not use trusted certificates
320-
# then you have to disable TLS validation
321-
# This is recommended for testing purposes only,
322-
# In production, use a QuestDB server certificate trusted
323-
# by your Kafka Connect installation
324-
tls.validation.mode=insecure
315+
# --- Source Kafka Topic ---
316+
topics=your_kafka_topic
325317

326-
# Rest of the Kafka config
327-
host=localhost
328-
topics=example-topics
329-
table=example-table
318+
# --- QuestDB Connection ---
319+
# Use https:: if your QuestDB server has TLS enabled.
320+
# Replace the placeholder with the token you generated.
321+
client.conf.string=https::addr=localhost:9000;token=qt1KAsf1U9YbUVAX1H2IahXEE3-4qBcK-zx_jsZUzV9bLY;
322+
323+
# --- Optional: Data Mapping ---
324+
# Use a field from the Kafka message key or value as a QuestDB symbol.
325+
# symbol.columns=device_id
330326
```
331327

332-
Can't connect? Check within your server logs.
328+
Once you deploy this configuration, the connector will start sending data from your Kafka topic to QuestDB. If you
329+
encounter any issues, check the logs for both your Kafka Connect worker and your QuestDB server for more details.
330+
331+
See the [QuestDB Kafka Connector documentation](/docs/third-party-tools/kafka/#questdb-kafka-connect-connector) for more details
332+
on the configuration options and how to set up the connector.
333333

334334
## 6. Query data, PostgreSQL query
335335

0 commit comments

Comments
 (0)