Skip to main content

PCAP Uploads

New in EveBox 0.32.0

PCAP uploads are only available in EveBox 0.32.0 and newer. The feature is still under active development and its configuration keys and behaviour may change.

EveBox can take a PCAP file uploaded from the browser, run Suricata on it and import the resulting events for review, then take the next one. This is the evebox oneshot --pcap flow driven from the web interface. It is always available in oneshot mode, and can be enabled on the EveBox Server, where uploaded captures are kept apart from live traffic.

Only classic pcap files are accepted. The pcapng format is not supported, as Suricata does not read it.

How It Works​

  1. The capture is sent to the server as a raw upload. The server checks that it starts with a classic pcap header and stays within the size limit as the bytes arrive, then queues it. Nothing is kept for a rejected file.
  2. One worker processes the queue, max-concurrent captures at a time and one by default. It runs Suricata on the capture, then imports the generated eve.json. Every event gets the upload's name as its host, so the capture appears in the sensor selector, labelled Capture, and host:<name> queries narrow to it. Each event also carries an evebox.upload object with the upload's id, name and filename.
  3. The original file is kept beside the events so that the packets of any of its events can be downloaded. See Packet Downloads.

Captures are named after their file, without the extension; a name that is already in use gets a numeric suffix. A name can also be given when uploading through the API.

Oneshot Mode​

In oneshot mode, the PCAP inputs given on the command line are processed as uploads, and the session keeps serving once they are done:

evebox oneshot --pcap capture1.pcap capture2.pcap

The Uploads page lists the captures as they are processed and accepts more for as long as the session runs. Starting with no inputs opens an empty session on the Uploads page:

evebox oneshot --pcap

--limit NUMBER caps the events imported from each capture. By default EveBox waits for the command-line inputs to be processed before it starts the server and opens the browser; with --no-wait it starts at once and the Uploads page shows the progress. The Suricata backend, container and rule options described in Try EveBox apply to every capture of the session, whether given on the command line or uploaded. The events, the processed captures and the temporary database are deleted when EveBox exits.

Server Mode​

On the EveBox Server, uploads are off by default and enabled with uploads.enabled: true in evebox.yaml; see Server Configuration. Uploaded captures are processed with Suricata on the server host, and their events are kept in a SQLite database of their own, uploads/uploads.sqlite under the data directory, never in the live datastore. This is the same whether the live datastore is SQLite, Elasticsearch or OpenSearch. The inbox, alerts, dashboards and live view never show uploaded events, and the live datastore's retention settings do not apply to them: captures are kept until they are deleted.

The web interface switches between the two. Uploads in the navigation bar opens the Uploads page and enters the uploads scope, in which the Events, Alerts and dashboard views show the uploaded captures, the sensor selector lists captures, and the Live view is hidden. A badge in the navigation bar shows that the uploads scope is active, with Back to live returning to live traffic. Archiving, escalating and commenting work on uploaded events as on live ones; this triage state belongs to the capture and is removed with it.

API clients select the store with the scope query parameter; see HTTP API.

The Uploads Page​

Drop PCAP files on the page or use Choose files to pick one or more. Files are checked in the browser before anything is sent: a file that is not a classic pcap, a pcapng file, or a file over the server's size limit is refused with the reason. Files are sent one at a time with a progress bar; a transfer can be cancelled, and files behind it wait their turn.

The table lists every upload with:

  • Status: queued, processing with the current phase, such as preparing or running Suricata, importing, ready, failed with the error beneath it, or deleting.
  • The event and alert counts, updated as the import runs, the time span of the traffic in the capture, who uploaded it and when, and the Suricata backend and version that processed it.
  • Actions: Events and Alerts open those views filtered to the capture; PCAP downloads the retained capture; Delete removes the capture's events, the retained file and the record, after confirmation. A capture still being processed can be deleted; its Suricata run is stopped.

The page refreshes itself, quickly while anything is queued, processing or being deleted.

Server Configuration​

Uploads are configured in the uploads section of evebox.yaml; there are no command-line or environment-variable equivalents. Every key but enabled has a default:

uploads:
enabled: true
# Largest accepted upload.
max-size: 4GiB
# Concurrent Suricata runs.
max-concurrent: 1
# Keep each capture on disk for packet downloads.
retain-pcap: true
suricata:
# auto (local Suricata when installed, otherwise a container), local
# or container.
backend: auto
path: /usr/bin/suricata
suricata-update: /usr/bin/suricata-update
container-runtime: auto # auto, podman or docker
image: docker.io/jasonish/suricata:8.0
KeyDefaultDescription
enabledfalseAccept uploads. When off, the Uploads page and the /api/uploads endpoints are absent and nothing is created on disk.
max-size4GiBLargest accepted upload; see Size Units. The browser refuses larger files before sending them.
max-concurrent1Captures processed at once. Suricata uses every core on its own, so one is usually right.
retain-pcaptrueKeep each capture under the data directory for packet downloads. With false, captures are removed once imported and their events have no packet downloads.
suricata.backendautolocal runs an installed Suricata; container runs Suricata in a Podman or Docker container. auto uses an installed Suricata when one is found, otherwise a container on Linux.
suricata.pathsuricata on PATHThe Suricata executable for the local backend. Setting it selects the local backend under auto.
suricata.suricata-updatesuricata-update on PATHThe suricata-update executable. Without one, rules are downloaded directly from the rule feeds.
suricata.container-runtimeautopodman or docker for the container backend; auto tries Podman first. Setting it selects the container backend under auto.
suricata.imagedocker.io/jasonish/suricata:8.0The Suricata container image, pulled when missing.

suricata.path and suricata.suricata-update cannot be combined with backend: container, nor suricata.container-runtime and suricata.image with backend: local; the server refuses to start and names the key. The captures, their events and the Suricata workspace all live under uploads/ in the server's data directory, which must be writable.

Size Units​

max-size takes a plain number of bytes, or a number with a unit. KiB, MiB, GiB and TiB, and the bare K, M, G and T, are binary units, so 4GiB and 4G are both 4,294,967,296 bytes. kB, MB, GB and TB are decimal: 4GB is 4,000,000,000 bytes. Units are not case sensitive, and a size of zero is rejected.

The same spellings are accepted by evebox pcap purge --max-size.

Requirements​

  • Suricata on the server host. The local backend needs Suricata 8.0.0 or newer, and uses suricata-update when it is installed. The container backend needs Podman or Docker on Linux, and pulls the image when it is missing. Both run Suricata with its stock configuration and the default HOME_NET.
  • Outbound network access for the rule downloads and, with the container backend, the image pull.
  • A writable data directory, where the captures and their events are kept.

Suricata is prepared when the first capture after the server starts is processed: the backend is selected and the ET/Open, PAW Patrules and The Hunters Ledger rules are downloaded, so that first capture takes noticeably longer, with the Uploads page showing the phase. Later captures reuse the prepared rules until the server restarts. The rule download cache is kept in the cache directory of the user the server runs as, so unchanged rules do not need to be downloaded again.

Security and Resources​

EveBox has no user roles, so any user who can log in may upload, and when authentication is disabled anyone who can reach the server may. uploads.enabled is the only gate. Each upload records the session username of the uploader.

Each upload costs CPU and disk on the server host. Suricata saturates the available cores while a capture is processed, and a capture needs working space for the generated eve.json until its import finishes. Bound the load with max-size and max-concurrent, turn off retain-pcap when packet downloads are not needed, and delete uploads that are no longer needed, as no retention applies to them.

Container Images​

The official EveBox container image ships neither Suricata nor a container engine, so uploads cannot be processed in it as is. Either build a derived image that adds Suricata (and suricata-update) and use the local backend, or give the EveBox container access to the host's Podman or Docker socket and use the container backend. In both cases the data directory must be a persistent volume, as it holds the captures and their events.

Packet Downloads​

Packets for an event from an uploaded capture are extracted from the retained capture itself; no capture spool or agent is involved. The source is shown as (uploads), which /api/pcap/sources and the custom download page's source picker list beside (server) and the connected agents. Like (server), the name is reserved: an agent cannot use it, and it serves events from uploaded captures only. Events from uploaded captures are served from it before any routing rule is considered, and rules cannot name it as a target.

With retain-pcap: false, or once a capture has been deleted, its events have no packet downloads. On Windows, extraction from retained captures needs Npcap, like every other local source.

HTTP API​

The upload endpoints live under /api/uploads and need a session when authentication is required. A capture is sent as the raw request body:

curl --fail -X POST 'http://localhost:5636/api/uploads?filename=capture.pcap' \
--data-binary @capture.pcap
Method and pathDescription
POST /api/uploadsUpload a capture; 202 Accepted with the new record. Optional filename and name query parameters.
GET /api/uploadsList every upload, newest first.
GET /api/uploads/{id}One upload, for polling its progress.
DELETE /api/uploads/{id}Delete an upload; 202 Accepted, and the deletion finishes in the background.
GET /api/uploads/{id}/pcapDownload the retained capture.

The event endpoints read the uploaded captures with ?scope=uploads, for example GET /api/events?scope=uploads. See the API reference for the record fields, the error codes and the scope parameter.

Limitations​

  • Classic pcap files only, uncompressed. pcapng is not supported, as Suricata does not read it.
  • Suricata runs with its stock configuration and the default rule sources; HOME_NET and the rules cannot be set per upload.
  • With the container backend, captures given on the evebox oneshot --pcap command line are copied into the container's mount directory and refused above 4 GiB unless --force is given. Uploaded captures are not copied; max-size is their only limit.
  • Uploaded events never appear in the Live view or on the firehose endpoints.