Docs
Elasticsearch Guide [8.1] » Release notes » Elasticsearch version 8.1.0
« Elasticsearch version 8.1.1 Elasticsearch version 8.0.1 »

Elasticsearch version 8.1.0

Also see Breaking changes in 8.1.

Breaking changes

Geo

Bug fixes

Aggregations
  • Reenable BooleanTermsIT #83421 (issue: #83351)
  • backward compatibility with version 7.17.0 #83715
  • ip prefix bucket reduction #83637
  • reduce float and half-float values to their stored precision #83213
Allocation
  • Fix updateMinNode condition #80403 (issue: #41194)
  • Make *.routing.allocation.* list-based setting #80420 (issue: #77773)
  • Permit metadata updates on flood-stage-blocked indices #81781
  • Reroute after cluster recovery #82856 (issue: #82456)
Authorization
  • Capture anonymous roles when creating API keys #81427 (issue: #81024)
  • Extend fleet-server service account privileges #82600
Autoscaling
  • Fix autoscaling of follower data streams #83302 (issue: #82857)
Client
  • Fix for self-suppression in REST client #83568 (issue: #42223)
Distributed
  • [GCE Discovery] Correctly handle large zones with 500 or more instances #83785 (issue: #83783)
Engine
  • Fork to WRITE thread when failing shard #84606 (issue: #84602)
Geo
  • Handle bounds properly when grid tiles crosses the dateline #83348 (issue: #83299)
  • GeometryNormalizer should not fail if it cannot compute signed area #84051 (issue: #83946)
ILM+SLM
  • Fix PolicyStepsRegistry's cachedSteps null handling #84588
Indices APIs
  • Allow removing unreferenced composable data stream templates #84376 (issues: #84171, #84188)
  • Simplify and speed up ExecutorSelector #83514 (issue: #82450)
Infra/Core
  • Always re-run Feature migrations which have encountered errors #83918 (issue: #83917)
  • Copy trace.id in threadcontext stash #83218
  • Preserve context in ResultDeduplicator #84038 (issue: #84036)
  • Registration of SystemIndexMigrationTask named xcontent objects #84192 (issue: #84115)
  • Update system index mappings if _meta is null #83896 (issue: #83890)
Infra/REST API
  • Do not allow safelisted media types on Content-Type #83448
Infra/Scripting
  • Fix duplicated allow lists upon script engine creation #82820 (issue: #82778)
  • Fix plumbing in double and keyword runtime fields for the scripting fields API #83392
Ingest
  • Fix GeoIpDownloader startup during rolling upgrade #84000
  • Short circuit date patterns after first match #83764
Machine Learning
  • Allow autoscaling to work when vertical scaling is possible #84242 (issue: #84198)
  • Correctly capture min stats for inference.ingest_processors in ML usage #82352
  • Fail queued inference requests with cause if the process crashes #81584
  • Fix NLP tokenization never_split handling around punctuation #82982
  • Fix ZeroShotClassificationConfig update mixing fields #82848
  • Fix bug where initial scale from 0→1 could scale too high #84244
  • Fix submit after shutdown in process worker service #83645 (issue: #83633)
  • Fixes categorize_text parameter validation to be parse order independent #82628 (issue: #82629)
  • Record node shutdown start time for each node #84355
  • Register the named X-content parser for snapshot upgrade params #84420 (issue: #84419)
  • Retry anomaly detection job recovery during relocation #83456
  • Return zxx for lang_ident_model_1 if no valid text is found for language identification #82746 (issue: #81933)
  • Text structure finder caps exclude lines pattern at 1000 characters #84236 (issue: #83434)
  • Validate vocabulary on model deployment #81548 (issue: #81470)
  • Wait for model process to stop in stop deployment #83644
  • Fix a bug in the tuning of the hyperparameters when training regression classification models #2128
  • Improve training stability for regression and classification models #2144, #2147, #2150
  • Avoid edge cases in the classification weights calculation to maximize minimum recall which could lead to only a single class being predicted #2194
  • Address cause of "[CStatisticalTests.cc@102] Test statistic is nan" log errors #2196
  • Address possible causes of "x = NaN, distribution = N5boost4math23students_t_distribution" log errors #2197
  • Fix bug restoring data gatherer state for time of day and week anomaly detection functions. This could lead to "No queue item for time" and "Time is out of range. Returning earliest bucket index" log errors #2213
Mapping
  • Add support for sub-fields to search_as_you_type fields #82430 (issue: #56326)
  • Better exception message for MappingParser.parse #80696
  • Completion field to support multiple completion multi-fields #83595 (issue: #83534)
Network
  • Throw NoSeedNodeLeftException on proxy failure #80961 (issue: #80898)
Packaging
  • Add log4j-slf4j-impl to repository-azure #83661 (issue: #83652)
  • Restart ES after keystore upgrade in postinst #84224 (issue: #82433)
Recovery
  • Add missing indices.recovery.internal_action_retry_timeout to list of settings #83354
  • Add missing max overcommit factor to list of (dynamic) settings #83350
SQL
  • Fix txt format for empty result sets #83376
Search
  • Avoid eagerly loading StoredFieldsReader in fetch phase #83693 (issue: #82777)
  • Do not deserialise the document when not needed in the fields fetch phase #84184
  • Returns valid PIT when no index matched #83424
Security
  • Add validation for API key role descriptors #82049 (issue: #67311)
Snapshot/Restore
  • Adjust LinuxFileSystemNatives.allocatedSizeInBytes for aarch64 architectures #81376 (issues: #80437, #81362)
  • Distinguish "missing repository" from "missing repository plugin" #82457 (issue: #81758)
  • Fix DirectBlobContainerIndexInput cloning method #84341 (issue: #84238)
  • Move get snapshots serialization to management pool #83215
  • Preserve context in snapshotDeletionListeners #84089 (issue: #84036)
TSDB
  • Fix time series timestamp meta missing #80695
Transform
  • Fix NPE in transform version check #81756
  • Fix condition on which the transform stops processing buckets #82852
  • Prevent stopping of transforms due to threadpool limitation #81912 (issue: #81796)
Watcher
  • Tolerate empty types array in Watch definitions #83524 (issue: #83235)

Deprecations

CRUD
  • Bulk actions JSON must be well-formed #78876 (issue: #43774)
Cluster Coordination
  • Remove last few mentions of Zen discovery #80410
SQL
  • Deprecate index_include_frozen request parameter #83943 (issue: #81939)

Enhancements

Aggregations
  • Add an aggregator for IPv4 and IPv6 subnets #82410
  • Fail shards early when we can detect a type missmatch #79869 (issue: #72276)
  • Optimize significant_text aggregation to only parse the field it requires from _source #79651
Allocation
  • Identify other node in SameShardAllocDec message #82890 (issue: #80767)
  • Make AllocationService#adaptAutoExpandReplicas Faster #83092
  • Speed up same host check #80767
Analysis
  • Expose Japanese completion filter to kuromoji analysis plugin #81858
Authentication
  • Enable run_as for all authentication schemes #79809
  • Return API key name in _authentication response #78946 (issue: #70306)
Authorization
  • Avoid loading authorized indices when requested indices are all concrete names #81237
  • Optimize DLS bitset building for matchAll query #81030 (issue: #80904)
Cluster Coordination
  • Add detail to slow cluster state warning message #83221
  • Batch Index Settings Update Requests #82896 (issue: #79866)
  • Improve node-join task descriptions #80090
  • Make PeerFinder log messages happier #83222
  • More compact serialization of metadata #82608 (issue: #77466)
  • Paginate persisted cluster state #78875
  • Reduce verbosity-increase timeout to 3 minutes #81118
  • Use network recycler for publications #80650 (issue: #80111)
Data streams
  • Defer reroute when autocreating datastream #82412 (issue: #82159)
ILM+SLM
  • Expose the index age in ILM explain output #81273 (issue: #64429)
Indices APIs
  • Batch auto create index cluster state updates #82159
  • Expose features option in Get Index API #83083 (issue: #82948)
  • Expose index health and status to the _stats API #81954 (issue: #80413)
  • Force merge REST API support wait_for_completion #80463 (issues: #80129, #80129)
Infra/Circuit Breakers
  • Allow dynamically changing the use_real_memory setting #78288 (issue: #77324)
Infra/Core
Infra/REST API
  • Allow for customised content-type validation #80906 (issue: #80482)
  • Update YAML REST tests to check for product header on all responses #83290
Infra/Scripting
  • Add $ syntax as a shortcut for field in Painless #80518
  • Add BinaryDocValuesField to replace BytesRef (ScriptDocValues) #79760
  • Add a geo point field for the scripting fields api #81395
  • Add date fields to the scripting fields api #81272
  • Add half float mapping to the scripting fields API #82294
  • Add scaled float to the scripting fields API #82275
  • Add support for GeoShape to the scripting fields API #81617
  • Fields API for IP mapped type #81396
  • Fields API for byte, double, float, integer, long, short #81126 (issue: #79105)
  • Fields API for flattened mapped type #82590
  • Fields API for x-pack constant_keyword #82292
  • Fields API for x-pack version, doc version, seq no, mumur3 #81476
  • Improve support for joda datetime to java datetime in Painless #83099
  • Keyword fields API support #81266
  • Make wildcard accessible from the scripting field API #82763
  • Ordinal field data plumbing #80970 (issue: #79105)
  • Support boolean fields in Fields API #80043 (issue: #79105)
  • Time series compile and cache evict metrics #79078 (issue: #62899)
Infra/Settings
  • Optimize duplicated code block in MetadataUpdateSettingsService #82048
Machine Learning
  • Add ability to update the truncation option at inference #80267
  • Add error counts to trained model stats #82705
  • Add latest search interval to datafeed stats #82620 (issue: #82405)
  • Adds new MPNet tokenization for NLP models #82234
  • Force delete trained models #80595
  • Improve error message on starting scrolling datafeed with no matching indices #81069 (issue: #81013)
  • Report thread settings per node for trained model deployments #81723 (issue: #81149)
  • Set default value of 30 days for model prune window #81377
  • Track token positions and use source string to tag NER entities #81275
  • Warn when creating job with an unusual bucket span #82145 (issue: #81645)
  • Improve skip_model_update rule behavior #2096
  • Prevent over-subscription of threads in pytorch_inference #2141
Mapping
  • Allow doc-values only search on geo_point fields #83395
  • Implement all queries on doc-values only keyword fields #83404
  • Optimize source filtering in SourceFieldMapper #81970 (issues: #77154, #81575)
Monitoring
  • Add Enterprise Search monitoring index templates #82743
  • Add beats_stats.metrics.apm-server.sampling.tail to Stack Monitoring templates #82401
Network
  • Report close connection exceptions at INFO #81768 (issues: #51612, #66473)
  • Serialize outbound messages on netty buffers #80111
  • Track histogram of transport handling times #80581 (issue: #80428)
Recovery
  • Adjust indices.recovery.max_bytes_per_sec according to external settings #82819
SQL
  • Compress Cursors #83591
  • Extend Tableau connector to reconnect with catalog #81321
Search
Security
  • Activate user profile API #82400
  • Add an initial ProfileService for user profiles #81899
  • Add new system index for user profile documents #81355
  • Add update user profile data API #82772
  • Add user profile API for get profile by UID #81910
  • Update Kibana system user privileges #82781
Snapshot/Restore
  • Add Linux x86-64bits native method to retrieve the number of allocated bytes on disk for a file #80437 (issue: #79698)
Stats
  • Add index pressure stats in cluster stats #80303 (issue: #79788)
  • Optimize getIndices in IndicesSegmentResponse #80064
  • Speed up MappingStats Computation on Coordinating Node #82830
TSDB
  • Add _tsid field to time_series indices #80276
  • Make time boundaries settings required in TSDB indices #81146
Transform
  • Introduce deduce_mappings transform setting #82256 (issue: #82559)
  • Make it possible to clear retention policy on an existing transform #82703 (issue: #82560)
  • Report transforms without config as erroneous #81141 (issue: #80955)

New features

Authentication
  • Initial version of JWT Realm #82175
  • Introduce domain setting to associate realms #81968
Distributed
  • Add desired nodes API #82975
Geo
  • New GeoHexGrid aggregation #82924
Health
  • Model for the new health reporting api #83398
TSDB
  • Handle fields.with.dots in routing_path #83148
Transform

Upgrades

Geo
  • Update vector tiles google protobuf to 3.16.1 #83402
Machine Learning
  • Upgrade Boost libraries to version 1.77 #2095
  • Upgrade RapidJSON to 31st October 2021 version #2106
  • Upgrade Eigen library to version 3.4.0 #2137
Network
  • Upgrade to Netty 4.1.73 #82844
Packaging
Security
  • Upgrade jANSI dependency to 2.4.0 #83566
« Elasticsearch version 8.1.1 Elasticsearch version 8.0.1 »