Quick start

From a built checkout to a queryable provenance graph.

On this page

1. 1. Bring up the backend
2. 2. Confirm the topics
3. 3. Point an agent at the stack
4. 4. Look at the graph
5. 5. Run a traversal
6. 6. Watch the fleet
7. Common failures

This page assumes you have completed Installation: the submodules are checked out, the solution builds, the certificates are on disk, and the environment variables are set.

1. Bring up the backend

docker compose up -d

Compose starts the broker first, then a one-shot redpanda-init container that creates the SASL users, the ACLs and the three topics before exiting, then the .NET services. A redpanda-init exiting non-zero is the usual cause when every downstream service fails at once.

docker compose ps
docker compose logs redpanda-init

2. Confirm the topics

Three topics should exist. Two carry events; one carries identity state.

Topic	Partitions	Cleanup	Retention
RawEvents	10	delete	1 day
ProcessedEvents	10	delete	1 day
IdentityChangelog	10	compact	infinite

The Redpanda Console on port 443 lists the topics, consumer groups and individual messages.

IdentityChangelog must have the same partition count as RawEvents. Entries are produced to the same partition as the event that caused them, so a mismatch makes a restoring node read the wrong slice of the log and rebuild an incorrect identity state.

3. Point an agent at the stack

For an initial test, run the agent from the build output on a Windows machine with EXTERNAL_IP reachable from it. For any longer-lived deployment, build a configured installer; see Deploying agents.

The agent needs, at minimum:

Kafka:
  BootstrapServers: broker:19092
  SchemaRegistryUrl: https://broker:18081
  SaslUsername: agent
  SaslPassword: "..."          # encrypted at rest once installed
EventSource:
  EnabledSources: [Process, FileIO, IP, DNS, Registry, Account]

The agent must run elevated. ETW real-time sessions and the Security event log are not readable by an unprivileged process, and the shipping agent runs as a LocalSystem Windows Service.

Watch it arrive:

docker compose logs -f provcollector.eventprocessor

The ThroughputMonitor UI on port 8083 shows live event rates per topic, and is the earliest confirmation that collection is working.

4. Look at the graph

Events reach PostgreSQL as two tables — nodes (one row per object per agent per day) and edges (one row per event). Both are PARTITION BY LIST (agent_id) with a day partition created on demand.

SELECT count(*) FROM nodes;
SELECT count(*) FROM edges;

5. Run a traversal

The Tracker CLI runs a time-respecting traversal per seed and writes each result as a NetworkX-compatible JSON graph:

ProvCollector.Tracker --csv seeds.csv --direction Backward --algorithm BFS \
    --output-dir Output --node-limit 50000

seeds.csv is id,label pairs; --csv - reads stdin.

A backward traversal from a frequently accessed file can reach most of the database. --node-limit and the --min-timestamp / --max-timestamp bounds are what keep such a query finite, and should be treated as required in practice.

For interactive exploration, you can open the JSON graph files generated by the Tracker CLI in the Visualizer on port 8080.

See the Tracker CLI reference for every option.

6. Watch the fleet

The AgentDashboard on port 8082 lists connected agents with their telemetry, lets you read and push configuration, issue commands, and upload installer packages for rollout. It holds no state of its own — every read and write goes through the fleet API on port 5080.

Common failures

Symptom	Likely cause
Broker will not start	Missing certificate under /opt/ProvCollector/redpanda-certs/
No agents appear in the dashboard	API_KEY_HASH unset, so FleetServer refuses every agent
Process nodes carry only a PID	Node carry-forward window mismatched — see Invariants