[Documentation] How can I enable python source logging to a mezmo log analysis destination?

[sabaimran]

I haven't found the documentation here to be very straightforward. I think I'm supposed to create a new pipeline in order to establish a destination (in this case, the mezmo log analysis destination). When I added the ingestion key to the logger (via LogDNAHandler) in my python code, it doesn't end up sending out any logs.

Not sure what my source is supposed to be in this pipeline. I originally added HTTP since that seemed to be the closest match for using a code library, but that doesn't seem right as the key doesn't serve any purpose.

Any guidance here is appreciated.

[dkhokhlov]

@sabaimran
By default this library is sending logs to standard logging endpoint - Log Analysis destination (not Mezmo Pipeline):
https://github.com/logdna/python/blob/master/logdna/configs.py#L15
Once you have free Mezmo account opened and got the KEY you can use this code example to verify that it is sending logs and you can see them in live view of Web UI:
https://github.com/logdna/python#setup
Let me know if your goal is to use Pipeline.

[sabaimran]

Awesome, thanks for the guidance @dkhokhlov . The goal is not to use pipeline -- I'd love to simply use it for Log Analysis. When I setup the script exactly as described in the setup snippet, I'm not able to actually see those logs anywhere. This is how I tested it:

(.venv) si@redwood logdna-test % python3
Python 3.11.4 (main, Jul 10 2023, 18:52:37) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import logging
>>> from logdna import LogDNAHandler
>>> key = "<REDACTED INGESTION KEY FROM MEZMO SETTINGS>
>>> logger = logging.getLogger('logdna')
>>> logger.setLevel(logging.INFO)
>>> test = LogDNAHandler(key)
>>> logger.addHandler(test)
>>> logger.info("test info message")

I don't see any errors, but I see a perpetual log-line from one of the underlying url libraries that indicates it's trying to connect when I run this in my application code. None of the logs seem to show up in the Mezmo dashboard.

[dkhokhlov]

Looks like as issue with KEY. Lets try sending sample log line with curl:

curl "https://logs.logdna.com/logs/ingest?hostname=MYHOST&now=$(date +%s)000" -u "XXXXXXXXXXXXXXXXXXXX:" --header "Content-Type: application/json; charset=UTF-8" -d "{\"lines\":[{\"line\":\"This is a sample test log statement\",\"timestamp\":\"2018-11-02T10:53:06+00:00\",\"level\":\"INFO\",\"app\":\"myapp\"}]}"

• replace XXXXX with your key. note terminating ":".

Expected:

{"batchID":"704a9733-5dac-4113-816e-bdb0a0fcda39:4400:ld81","status":"ok"}

and you should see the sample line in live view.

full description of this method is here:
https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-ingest

[davidkhala]

This is more critical than that, The python sdk does not authenticate the provided key before sending request. And no error will be raised if trying to send log with a handler configured with a invalid ingestion key. The False-Negative behavior indicates a software quality issue.

Only a message Please provide a valid ingestion key. Discarding flush buffer will to flush to stdout.