doc/ai/module_map.yaml - main - githubmirror/rsyslog

doc/ai/module_map.yaml
194 строки8 KB
Rainer Gerhards
mmnormalize: make HUP reload worker-safe
25 июл 2026, 20:53
25 июл 2026, 20:5380be65d
# Seed map for AI agents; keep minimal and truthful. Extend over time.
# Per-module metadata (support status, maturity, contacts) lives in each module
# directory as `MODULE_METADATA.yaml`; update both the metadata file and this map
# when concurrency guidance changes.
runtime_queue:
  paths:
    - "runtime/queue.c"
    - "runtime/queue.h"
    - "runtime/queue_da.c"
    - "runtime/queue_da.h"
    - "runtime/segdisk_store.c"
    - "runtime/segdisk_store.h"
  requires_serialization: true
  notes:
    - Disk-assisted mode is an in-memory parent plus a first-class disk child; disk queue semantics apply to the child.
    - The queue mutex is shared by a disk-assisted parent and child; segmented store operations and idle dematerialization callbacks run under that lock.
    - queue_da resolves the durable disk-child engine at startup and atomically publishes its marker before first persistent data; engines never switch while a process is running.
    - The segmented store is lazy only for a disk-assisted child, and dematerialization requires an empty store with no recovery, batch, retry, or completion work pending.

runtime_ratelimit:
  paths: ["runtime/ratelimit.c", "runtime/ratelimit.h"]
  requires_serialization: true
  locks:
    - type: rwlock
      field: ratelimit_cfg_bucket_t.lock
    - type: mutex
      field: ratelimit_shared_t.mut
    - type: mutex
      field: ratelimit_shared_t.per_source_policy_mut
    - type: mutex
      field: ratelimit_shared_t.per_source_key_policy_mut
    - type: mutex
      field: ratelimit_ps_bucket_t.mut
    - type: mutex
      field: ratelimit_t.mut
  notes:
    - Named ratelimit configs are stored in sharded hashtables; lookup and insertion lock only the shard selected by the config name.
    - Input per-source ratelimit state is stored in shard-local hashtables with embedded override data; message-time accounting locks only the source-key shard.
    - Per-source key policy uses a versioned atomic snapshot of the template pointer and packed mode flags; readers retry across updates, and the key-policy mutex is only the no-atomics fallback.
    - perSourceMaxStates is enforced approximately through shard-local caps; values below the shard count keep one active state per shard, and per_source_evicted reports source-state evictions.
    - HUP snapshots named configs under shard read locks, then reloads policy files without holding registry locks.

runtime_tcpsrv:
  paths:
    - "runtime/tcpsrv.c"
    - "runtime/tcpsrv.h"
    - "runtime/tcps_sess.c"
    - "runtime/tcps_sess.h"
  requires_serialization: false
  notes:
    - imtcp sessions own independent decompressor state.
    - A finite zstd aggregate-window limit uses a listener-local 64-bit atomic reservation counter; only platforms without 64-bit atomics use the helper-mutex fallback.

runtime_msg:
  paths: ["runtime/msg.c", "runtime/msg.h"]
  requires_serialization: true
  notes:
    - MsgGetRcvFromProp returns fromhost, fromhost-ip, or fromhost-port with the stored prop length; it resolves DNS only when NEEDS_DNSRESOL is still set.

mmnormalize:
  paths: ["plugins/mmnormalize/"]
  requires_serialization: true
  locks:
    - type: rwlock
      field: instanceData.ctxlnLock
    - type: atomic-generation
      field: instanceData.reloadGeneration
  notes:
    - The standard liblognorm context is shared; actions take the read lock and HUP publishes a privately built replacement under the write lock.
    - Turbo contexts are worker-owned and remain lock-free while normalizing; HUP increments a generation and each worker rebuilds and reclaims its own context between messages.

omfile:
  paths: ["plugins/omfile/"]
  requires_serialization: true
  locks:
    - type: mutex
      field: pData.mutWrite

omfwd:
  paths: ["plugins/omfwd/"]
  requires_serialization: false

omelasticsearch:
  paths: ["plugins/omelasticsearch/"]
  requires_serialization: false

omazureeventhubs:
  paths: ["plugins/omazureeventhubs/"]
  requires_serialization: true
  locks:
    - type: rwlock
      field: pData.pnLock
      writer: rsyslog_worker
      reader: proton_thread

omazuredce:
  paths: ["plugins/omazuredce/"]
  requires_serialization: true
  locks:
    - type: mutex
      field: pWrkrData.batchLock
  notes:
    - Each worker owns its batch buffer and timer thread; access is serialized by batchLock.

omawslogshlc:
  paths: ["plugins/omawslogshlc/"]
  requires_serialization: false
  notes:
    - Each worker owns its CURL handle and HLC batch buffer; action configuration is immutable after startup.

contrib_omhttp:
  paths: ["contrib/omhttp/"]
  requires_serialization: false

contrib_mmkubernetes:
  paths: ["contrib/mmkubernetes/"]
  requires_serialization: false
  notes:
    - WID is per-thread; shared caches in pData must be guarded.

contrib_imkubernetes:
  paths: ["contrib/imkubernetes/"]
  requires_serialization: false
  notes:
    - The single input thread owns file tail state, partial-record buffers, cache entries, and the CURL client.

mmsnareparse:
  paths: ["plugins/mmsnareparse/"]
  requires_serialization: false
  notes:
    - Parsing logic does not maintain shared mutable state.

ommysql:
  paths: ["plugins/ommysql/"]
  requires_serialization: false
  notes:
    - Per-worker MYSQL handle in pWrkrData; no module-level RW lock required after refactor.

mmjsontransform:
  paths: ["plugins/mmjsontransform/"]
  requires_serialization: false
  notes:
    - Rewrites JSON trees per message without shared mutable state.

mmjsonrewrite:
  paths: ["plugins/mmjsonrewrite/"]
  requires_serialization: false
  notes:
    - Expands dotted JSON property names into nested containers per message.

mmexternal:
  paths: ["plugins/mmexternal/"]
  requires_serialization: true
  locks:
    - type: mutex
      field: pData.mut
  notes:
    - Default mode keeps one child process context per worker.
    - forceSingleInstance shares one child context in pData and serializes helper access with pData.mut.

omotel:
  paths: ["plugins/omotel/"]
  requires_serialization: false
  notes:
    - Transport resources are not yet active; introduce pData locking once they appear.

imbeats:
  paths: ["plugins/imbeats/"]
  requires_serialization: true
  locks:
    - type: mutex
      field: instanceConf.mutSessions
  notes:
    - Listener threads own readiness notification and worker threads process native Lumberjack session state.
    - Use Elastic Agent or Filebeat output.logstash to send Lumberjack v2 traffic to imbeats.
    - Concurrent Beats sessions are bounded per listener with maxSessions and processed through the listener worker/multiplexing path.
    - Use starvationProtection.maxReads to keep one busy sender from monopolizing imbeats worker time.
    - maxInFlightBytes accounts incomplete receive buffers, inflate scratch space, event copies, and lazy window descriptors exactly across each input instance.
    - maxCompressionRatio and maxDecompressedSize jointly bound compressed-frame expansion.
    - idleTimeout closes sessions with no byte progress; frameTimeout bounds one frame and windowTimeout bounds one advertised window with absolute deadlines that trickle traffic cannot extend. These network deadlines pause while a complete batch is submitted and resume for its ACK. A value of 0 disables the corresponding timeout.
    - JSON event payloads must be exactly one object, apart from surrounding whitespace. Malformed, non-object, trailing-content, resource-limited, or timed-out traffic is not acknowledged.
    - Valid Lumberjack v2 W, J, and C flows retain cumulative A acknowledgement behavior.
    - TLS deployments require an installed rsyslog TLS stream-driver package or module, such as rsyslog-gnutls or rsyslog-openssl.
    - The common Elastic Agent TLS setup validates the rsyslog server certificate but does not present a client certificate; use streamDriver.authMode=anon for that pattern and stricter auth modes only with sender client certificates.
    - Compressed Lumberjack v2 frames are supported and should be tested with Beats compression enabled.
    - A concrete sample container definition lives under packaging/docker/rsyslog/imbeats but is not wired into published container builds yet.

imfifo:
  paths: ["plugins/imfifo/"]
  requires_serialization: true
  notes:
    - Reads log messages line-by-line from local POSIX named pipes (FIFOs).
    - Utilizes non-blocking poll() loops with a 100ms timeout to ensure clean and immediate shutdown.
    - Uses O_RDWR opening technique to keep a dummy writer in the daemon itself to avoid startup hangs and EOF reopening loops.