DatattentionLogs to keep before you open an inference port
The minimum record that lets you explain a model service after the fact.
If a model port accepts traffic from more than one machine, log enough to reconstruct a request without storing every prompt forever. The useful fields are the time, source address, method, path, model name, status, and how long the call took. Those fit in an access log.
Decide on purpose whether the prompt body is in scope. Some teams need it for abuse review. Many do not, and a disk full of prompts is a new store of sensitive text. If you keep bodies, set a size cap and a retention period, and say both in the same document as the port number.
Send the log off the machine that serves the model. A disk on that machine disappears with the machine. A second copy, shipped outbound on a schedule, is enough. You do not need a public log viewer. An admin interface for logs is another service to lock.
Clock skew makes the record useless. Sync time, and write timestamps in UTC. Include the host name so a later reader can tell two boxes apart.
Read the log once before you announce the service. The test is whether you can answer “who called the chat route yesterday” without logging into the GPU machine and guessing.
On this host
A request to any of the ports on this host is the record those fields describe. No API key is required. This chat call against port 8080 is enough to see a full OpenAI-style body land in the log.
curl http://datattention.com:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Hello"}]}'The full list of ports and paths is on the endpoints page.