Affecting all Beats

• Upgrade to Go 1.21.7. Removes support for Windows 8.1. See https://tip.golang.org/doc/go1.21#windows. 37913

• add_cloud_metadata processor: huawei provider is now treated as openstack. Huawei cloud runs on OpenStack platform, and when viewed from a metadata API standpoint, it is impossible to differentiate it from OpenStack. If you know that your deployments run on Huawei Cloud exclusively, and you wish to have cloud.provider value as huawei, you can achieve this by overwriting the value using an add_fields processor. 35184

• In managed mode, Beats running under Elastic Agent will report the package version of Elastic Agent as their own version. This includes all additional fields added to events containing the Beats version. 37553

• The behavior of queue.mem.flush.min_events has been simplified. It now serves as a simple maximum on the size of all event batches. There are no longer performance implications in its relationship to bulk_max_size. 37795

Auditbeat

• Add opt-in KProbes backend for file_integrity module. 37796

Filebeat

• Convert netflow input to API v2 and disable event normalisation 37901

Heartbeat

Metricbeat

Osquerybeat

Packetbeat

Winlogbeat

• Add "event.category" and "event.type" to Sysmon module for EventIDs 8, 9, 19, 20, 27, 28, 255 35193

• Add "keystore.path" configuration settings to $workdir\data\{{.BeatName}}.keystore. Issue 12315 37237

Functionbeat

Elastic Logging Plugin

Affecting all Beats - Support for multiline zookeeper logs 2496 - Add checks to ensure reloading of units if the configuration actually changed. 34346 - Fix namespacing on self-monitoring 32336 - Fix namespacing on self-monitoring 32336 - Fix Beats started by agent do not respect the allow_older_versions: true configuration flag 34227 34964 - Fix performance issues when we have a lot of inputs starting and stopping by allowing to disable global processors under fleet. 35000 35031 - 'add_cloud_metadata' processor - add cloud.region field for GCE cloud provider - 'add_cloud_metadata' processor - update azure metadata api version to get missing cloud.account.id field - Upgraded apache arrow library used in x-pack/libbeat/reader/parquet from v11 to v12.0.1 in order to fix cross-compilation issues 35640 - Fix panic when MaxRetryInterval is specified, but RetryInterval is not 35820 - Support build of projects outside of beats directory 36126 - Support Elastic Agent control protocol chunking support 37343 - Upgrade elastic-agent-libs to v0.7.5. Removes obsolete "Treating the CommonName field on X.509 certificates as a host name…​" deprecation warning for 8.0. 37755 - aws: Add credential caching for AssumeRole session tokens. 37787 - Lower logging level to debug when attempting to configure beats with unknown fields from autodiscovered events/environments 37816[37816] - Set timeout of 1 minute for FQDN requests 37756 - Fix the paths in the .cmd script added to the path by the Windows MSI to point to the new C:\Program Files installation location. https://github.com/elastic/elastic-stack-installers/pull/238

Auditbeat

Filebeat

• Fix nil pointer dereference in the httpjson input 37591

• [Gcs Input] - Added missing locks for safe concurrency 34914

• Fix the ignore_inactive option being ignored in Filebeat’s filestream input 34770

• Fix TestMultiEventForEOFRetryHandlerInput unit test of CometD input 34903

• Add input instance id to request trace filename for httpjson and cel inputs 35024

• Fixes "Can only start an input when all related states are finished" error when running under Elastic-Agent 35250 33653

• [system] sync system/auth dataset with system integration 1.29.0. 35581

• [GCS Input] - Fixed an issue where bucket_timeout was being applied to the entire bucket poll interval and not individual bucket object read operations. Fixed a map write concurrency issue arising from data races when using a high number of workers. Fixed the flaky tests that were present in the GCS test suit. 35605

• Fixed concurrency and flakey tests issue in azure blob storage input. 35983 36124

• Fix panic when sqs input metrics getter is invoked 36101 36077

• Fix handling of Juniper SRX structured data when there is no leading junos element. 36270 36308

• Fix Filebeat Cisco module with missing escape character 36325 36326

• Added a fix for Crowdstrike pipeline handling process arrays 36496

• Fix m365_defender cursor value and query building. 37116

• Fix TCP/UDP metric queue length parsing base. 37714

• Update github.com/lestrrat-go/jwx dependency. 37799

• [threatintel] MISP pagination fixes 37898

• Fix file handle leak when handling errors in filestream 37973

Heartbeat

Metricbeat

Osquerybeat

Packetbeat

• Fix interface device parsing for packetbeat protocols. 37946

Winlogbeat

Elastic Logging Plugin

Affecting all Beats

• Added append Processor which will append concrete values or values from a field to target. 29934 33364

• dns processor: Add support for forward lookups (A, AAAA, and TXT). 11416 36394

• [Enhanncement for host.ip and host.mac] Disabling netinfo.enabled option of add-host-metadata processor 36506 Setting environmental variable ELASTIC_NETINFO:false in Elastic Agent pod will disable the netinfo.enabled option of add_host_metadata processor

• allow queue configuration settings to be set under the output. 35615 36788

• Beats will now connect to older Elasticsearch instances by default 36884

• Raise up logging level to warning when attempting to configure beats with unknown fields from autodiscovered events/environments

• elasticsearch output now supports idle_connection_timeout. 35615 36843

• Upgrade golang/x/net to v0.17.0. Updates the publicsuffix table used by the registered_domain processor. 36969 Setting environmental variable ELASTIC_NETINFO:false in Elastic Agent pod will disable the netinfo.enabled option of add_host_metadata processor

• The Elasticsearch output can now configure performance presets with the preset configuration field. 37259

• Upgrade to elastic-agent-libs v0.7.3 and golang.org/x/crypto v0.17.0. 37544

• Make more selective the Pod autodiscovery upon node and namespace update events. 37338 37431

• Upgrade go-sysinfo from 1.12.0 to 1.13.1. 37996

Auditbeat

• Add linux capabilities to processes in the system/process. 37453

• Add opt-in eBPF backend for file_integrity module. 37223

Filebeat

• Update SQL input documentation regarding Oracle DSNs 37590

• add documentation for decode_xml_wineventlog processor field mappings. 32456

• httpjson input: Add request tracing logger. 32402 32412

• Add cloudflare R2 to provider list in AWS S3 input. 32620

• Add support for single string containing multiple relation-types in getRFC5988Link. 32811

• Added separation of transform context object inside httpjson. Introduced new clause .parent_last_response.* 33499

• Added metric sqs_messages_waiting_gauge for aws-s3 input. 34488

• Add nginx.ingress_controller.upstream.ip to related.ip 34645 34672

• Add unix socket log parsing for nginx ingress_controller 34732

• Added metric sqs_worker_utilization for aws-s3 input. 34793

• Add MySQL authentication message parsing and related.ip and related.user fields 34810

• Add nginx ingress_controller parsing if one of upstreams fails to return response 34787

• Add oracle authentication messages parsing 35127

• Add clean_session configuration setting for MQTT input. 16204

• Add support for a simplified input configuraton when running under Elastic-Agent 36390

• Added support for Okta OAuth2 provider in the CEL input. 36336 36521

• Added support for new features & removed partial save mechanism in the Azure Blob Storage input. 35126 36690

• Added support for new features and removed partial save mechanism in the GCS input. 35847 36713

• Re-use buffers to optimise memory allocation in fingerprint mode of filestream 36736

• Allow http_endpoint input to receive PUT and PATCH requests. 36734

• Add cache processor. 36786

• Avoid unwanted publication of Azure entity records. 36753

• Avoid unwanted publication of Okta entity records. 36770

• Add support for Digest Authentication to CEL input. 35514 36932

• Use filestream input with file_identity.fingerprint as default for hints autodiscover. 35984 36950

• Add network processor in addition to interface based direction resolution. 37023

• Add setup option --force-enable-module-filesets, that will act as if all filesets have been enabled in a module during setup. 30915 99999

• Make CEL input log current transaction ID when request tracing is turned on. 37065

• Made Azure Blob Storage input GA and updated docs accordingly. 37128

• Add request trace logging to http_endpoint input. 36951 36957

• Made GCS input GA and updated docs accordingly. 37127

• Suppress and log max HTTP request retry errors in CEL input. 37160

• Prevent CEL input from re-entering the eval loop when an evaluation failed. 37161

• Update CEL extensions library to v1.7.0. 37172

• Add support for complete URL replacement in HTTPJSON chain steps. 37486

• Add support for user-defined query selection in EntraID entity analytics provider. 37653

• Update CEL extensions library to v1.8.0 to provide runtime error location reporting. 37304 37718

• Add request trace logging for chained API requests. 36551 37682

• Relax TCP/UDP metric polling expectations to improve metric collection. 37714

• Add support for PEM-based Okta auth in HTTPJSON. 37772

• Prevent complete loss of long request trace data. 37826 37836

• Added experimental version of the Websocket Input. 37774

• Add support for PEM-based Okta auth in CEL. 37813

• Add ETW input. 36915

• Update CEL mito extensions to v1.9.0 to add keys/values helper. 37971

• Add logging for cache processor file reads and writes. 38052

Auditbeat

Libbeat - Add watcher that can be used to monitor Linux kernel events. 37833

• Added support for ETW reader. 36914

Heartbeat - Added status to monitor run log report. - Upgrade github.com/elastic/go-elasticsearch/v8 to v8.12.0. 37673

Metricbeat

• Add per-thread metrics to system_summary 33614

• Add GCP CloudSQL metadata 33066

• Add GCP Carbon Footprint metricbeat data 34820

• Add event loop utilization metric to Kibana module 35020

• Fix containerd metrics grouping for TSDB 37537

• Add metrics grouping by dimensions and time to Azure app insights 36634

• Align on the algorithm used to transform Prometheus histograms into Elasticsearch histograms 36647

• Enhance GCP billing with detailed tables identification, additional fields, and optimized data handling. 36902

• Add a /inputs/ route to the HTTP monitoring endpoint that exposes metrics for each metricset instance. 36971

• Add linux IO metrics to system/process 37213

• Add new memory/cgroup metrics to Kibana module 37232

Metricbeat

• Update getOpTimestamp in replstatus to fix sort and temp files generation issue in mongodb. 37688

Osquerybeat

Packetbeat

• Bump Windows Npcap version to v1.79. 37733

• Add metrics for TCP flags. 36992 36975

• Add support for pipeline loading. 37291

Packetbeat

Winlogbeat

Functionbeat

Winlogbeat

Elastic Log Driver Elastic Logging Plugin

Auditbeat

Filebeat

Heartbeat

Metricbeat

Osquerybeat

Packetbeat

Winlogbeat

Functionbeat

Elastic Logging Plugin