@@ -184,152 +184,152 @@ List the full list of applied permissions with `all_permissions()`.
184184
185185## 4. Ingest data, InfluxDB Line Protocol
186186
187- Perform data ingestion through a service account or an interactive user. Service
188- accounts are recommended over users, as they apply a cleaner set of access
189- permissions, and are less likely to be affected by day-to-day user management
190- operations.
187+ The recommended method for high-throughput ingestion is InfluxDB Line Protocol (ILP) over HTTP.
191188
192- First, set up the service account. Then, use it to create a token which is
193- associated with the service account. This token is then provided to your
194- InfluxDB client to form a secure, access-controlled connection.
189+ We recommend using a service account for programmatic ingestion. Service accounts apply a cleaner set of access
190+ permissions and are less likely to be affected by day-to-day user management.
195191
196- A service account is "an account for a service". This is in contrast to an
197- account for a user. When service accounts are created, we assume that they
198- belong to an organization and not an individual. Your sensors, apps or cars may
199- use service accounts. Sam, the plucky analyst, may have an interactive user.
192+ The process is:
200193
201- The recommended ingestion method is via the InfluxDB Line Protocol (ILP).
194+ 1 . Create a service account and grant it permissions.
195+ 2 . Generate a ** REST token** for the service account.
196+ 3 . Use this token in your client's connection string.
202197
203- To setup a service account:
198+ ### Step 1: Create the Service Account
199+
200+ First, run the following SQL in the web console. This creates a service account named ` ingest_http ` and grants it the
201+ necessary permissions to use HTTP endpoints and manage data.
204202
205203``` questdb-sql title="Web Console - Setup a service account"
206- CREATE SERVICE ACCOUNT ingest;
207- GRANT ilp, create table TO ingest;
208- GRANT add column, insert ON all tables TO ingest;
209- -- OR
210- GRANT add column, insert ON table1, table2 TO ingest;
204+ CREATE SERVICE ACCOUNT ingest_ilp;
205+ -- Grant permission to create tables and use HTTP endpoints
206+ GRANT HTTP, CREATE TABLE TO ingest_ilp;
207+ -- Grant permission to add columns and insert data
208+ GRANT ADD COLUMN, INSERT ON ALL TABLES TO ingest_ilp;
209+
210+ -- OR, for more granular control:
211+ -- GRANT ADD COLUMN, INSERT ON table1, table2 TO ingest_ilp;
211212```
212213
213- This creates a service account called ` ingest ` , which:
214-
215- - Can create a table
216- - Add table columns
217- - Insert to all tables OR insert to specific tables
214+ ### Step 2: Generate an Authentication Token
218215
219- The account exists, and that means that a client of some kind can connect to
220- QuestDB via that account. Next, create a token to create a secure link between
221- the client and the account:
216+ Next, generate a REST API token for the service account. This token acts as a password, so you must store it securely.
222217
223- ``` questdb-sql title="Web Console - Generate a token for ingest client"
224- ALTER SERVICE ACCOUNT ingest CREATE TOKEN TYPE JWK ;
218+ ``` questdb-sql title="Web Console - Generate a token for the ingest client"
219+ ALTER SERVICE ACCOUNT ingest_ilp CREATE TOKEN TYPE REST WITH TTL '3000d' REFRESH ;
225220```
226221
227- This creates a token comprised of three parts:
228-
229- - public_key_x
230- - public_key_y
231- - private_key
232-
233- | name | public_key_x | public_key_y | private_key |
234- | ------ | ------------------------------------------ | ------------------------------------------ | ------------------------------------------- |
235- | ingest | gxVbx90=MtYMmIEek2L5jFa5e9qTIvxI2TKSfI3GVE | kEUZjIfU9=S6w6uR=j130v003YgB3NBpYcVswvvacs | kom7j38LG44HcPfO92oZ4558e6KoeTHn6H5rA8vK3PQ |
236-
237- Now, this private key is then added to the client.
222+ This command returns a token. ** Copy it immediately** , as it's shown only once.
238223
239- This provides authenticated access to QuestDB for the "ingest" user.
224+ | name | token | expires_at | refresh |
225+ | ------------| ------------------------------------------------| -----------------------------| ---------|
226+ | ingest_ilp | qt1KAsf1U9YbUVAX1H2IahXEE3-4qBcK-zx_jsZUzV9bLY | 2033-09-19T15:32:51.628453Z | true |
240227
241- For example, if you are leveraging Java and our recommended InfluxDB Line Protocol over HTTP client:
228+ ### Step 3: Use the Token in Your Client
242229
243- ``` java
244- Java client example :
230+ You can now use this token to authenticate your application. The following Java example shows how to use the client
231+ library by configuring it from a connection string. This is the recommended approach.
245232
233+ ``` java title="Java - Ingesting data via ILP"
246234import io.questdb.client.Sender ;
247235import java.time.temporal.ChronoUnit ;
248236
249- public class ILPMain {
237+ public class Ingest {
250238 public static void main (String [] args ) {
251- try (Sender sender = Sender . builder(Sender . Transport . HTTP )
252- .address(" localhost:9000" )
253- .enableTls()
254- .enableAuth(" ingest" )
255- .authToken(" kom7j38LG44HcPfO92oZ4558e6KoeTHn6H5rA8vK3PQ" )
256- .build()) {
257-
239+ try (Sender sender = Sender . fromConfig(" https::addr=localhost:9000;token=qt1KAsf1U9YbUVAX1H2IahXEE3-4qBcK-zx_jsZUzV9bLY;" )) {
258240 sender. table(" ilptest" );
259241 sender. symbol(" sym1" , " symval1" )
260- .doubleColumn(" double1" , 100.0 )
261- .at(System . currentTimeMillis(), ChronoUnit . MILLIS );
242+ .doubleColumn(" double1" , 100.0 )
243+ .at(System . currentTimeMillis(), ChronoUnit . MILLIS );
262244 }
263245 }
264246}
265247```
266248
267- Please note that the private key is not stored in the database.
268-
269- There is ** no way to get your private key back ** at a later time!
270-
271- Once generated, safely store it.
249+ ::: note A Note on TLS
250+ The ` https:: ` prefix in the connection string tells the client to connect using TLS. By default, the client will verify
251+ the server's certificate. For local testing with self-signed certificates, you can disable this validation by adding
252+ ` tls.verify=insecure; ` to the configuration string. ** This is not recommended for production. **
253+ :::
272254
273255Connecting a client to ILP is a common path.
274256
275257However, you may use something like [ Kafka] ( /docs/third-party-tools/kafka ) .
276258
277259## 5. Ingest data, Kafka Connect (optional)
278260
279- _ If you're not applying Kafka, skip to step 6._
261+ _ If you're not using Kafka, you can skip to section 6._
280262
281- The
282- [ Kafka Connect] ( https://docs.confluent.io/platform/current/connect/index.html )
283- connector can be thought of as a specialized ILP client.
263+ The official ** QuestDB Kafka Connect sink** forwards messages from Kafka topics directly to your database using ILP protocol.
264+ The setup process is straightforward:
284265
285- Thus the steps are similar to ILP ingestion:
266+ 1 . Create a dedicated service account in QuestDB.
267+ 2 . Generate an authentication token for the account.
268+ 3 . Configure the Kafka sink connector with your QuestDB address and the token.
286269
287- 1 . Create a Kafka service account and assign permissions
288- 1 . Configure the connector to use the service account
270+ ### ** Step 1: Create the Service Account**
289271
290- ``` questdb-sql title="Web Console - Create a service account and grant required permissions"
272+ In the QuestDB web console, create a service account named ` kafka ` and grant it the permissions required to connect and
273+ write data.
274+
275+ ``` questdb-sql title="Web Console - Create a Kafka service account"
291276CREATE SERVICE ACCOUNT kafka;
292- GRANT ilp, create table TO kafka;
293- GRANT add column, insert ON all tables TO kafka;
294- OR
295- GRANT add column, insert ON table1, table2 to kafka;
277+
278+ -- Grant permissions to use HTTP, create tables, add new columns and insert data
279+ GRANT HTTP, CREATE TABLE TO kafka;
280+ GRANT ADD COLUMN, INSERT ON ALL TABLES TO kafka;
281+
282+ -- OR, for more granular control:
283+ -- GRANT ADD COLUMN, INSERT ON table1, table2 TO ingest_ilp;
296284```
297285
286+ ### ** Step 2: Generate an Authentication Token**
287+
288+ Next, generate a REST API token for the ` kafka ` service account. This token is a secret credential and should be treated like a
289+ password.
290+
298291``` questdb-sql title="Web Console - Generate a token for the service account"
299- ALTER SERVICE ACCOUNT kafka CREATE TOKEN TYPE JWK;
292+ -- Creates a token that is valid for 1 year (365 days)
293+ ALTER SERVICE ACCOUNT kafka CREATE TOKEN TYPE REST WITH TTL '365d';
300294```
301295
302- The token SQL returns a multi-part token, as before:
296+ The command returns a token. ** Copy it immediately ** , as it will not be shown again.
303297
304- | name | public_key_x | public_key_y | private_key |
305- | ----- | ----------------------------------------- | ------------------------------------------- | --------------------------------------------- |
306- | kafka | uE3MxG5_PMTor0V40LIBNTaUv-xh0dMPRN83nsQUI | 73wq8nx02Pj6W6yt53VxFT9K-TnopdM0s0UeeBxTgb0 | tDNC3DJ_L_QYIZRu_L4S4YTZYZXbjr7JX_bxFYdHhhhQU |
307-
308- Remember, the private key cannot be retrieved.
298+ | name | token | expires\_ at |
299+ | -------| --------------------------------------------------| -------------------------------|
300+ | kafka | ` qt1KAsf1U9YbUVAX1H2IahXEE3-4qBcK-zx_jsZUzV9bLY ` | ` 2026-07-03T18:05:00.000000Z ` |
309301
310302Save the private key in a secure location!
311303
312- Next, configure the username, token and TLS settings inside of Kafka:
304+ ### ** Step 3: Configure the Kafka Connect Sink**
305+
306+ Create a configuration file for the QuestDB sink connector. In the ` client.conf.string ` property, provide your QuestDB
307+ server address and paste the token you just generated.
313308
314- ``` bash title="Kafka Connect - Configuring the QuestDB Kafka connector"
315- username=kafka
316- token=[the private key saved in the previous step]
317- tls=true
309+ ``` properties title="questdb-sink.properties"
310+ # --- Connector Identity ---
311+ name =QuestDBSinkConnector
312+ connector.class =io.questdb.kafka.QuestDBSinkConnector
313+ tasks.max =1
318314
319- # If QuestDB server does not use trusted certificates
320- # then you have to disable TLS validation
321- # This is recommended for testing purposes only,
322- # In production, use a QuestDB server certificate trusted
323- # by your Kafka Connect installation
324- tls.validation.mode=insecure
315+ # --- Source Kafka Topic ---
316+ topics =your_kafka_topic
325317
326- # Rest of the Kafka config
327- host=localhost
328- topics=example-topics
329- table=example-table
318+ # --- QuestDB Connection ---
319+ # Use https:: if your QuestDB server has TLS enabled.
320+ # Replace the placeholder with the token you generated.
321+ client.conf.string =https::addr =localhost:9000; token=qt1KAsf1U9YbUVAX1H2IahXEE3-4qBcK-zx_jsZUzV9bLY;
322+
323+ # --- Optional: Data Mapping ---
324+ # Use a field from the Kafka message key or value as a QuestDB symbol.
325+ # symbol.columns=device_id
330326```
331327
332- Can't connect? Check within your server logs.
328+ Once you deploy this configuration, the connector will start sending data from your Kafka topic to QuestDB. If you
329+ encounter any issues, check the logs for both your Kafka Connect worker and your QuestDB server for more details.
330+
331+ See the [ QuestDB Kafka Connector documentation] ( /docs/third-party-tools/kafka/#questdb-kafka-connect-connector ) for more details
332+ on the configuration options and how to set up the connector.
333333
334334## 6. Query data, PostgreSQL query
335335
0 commit comments