With 7.0, Elasticsearch switched from joda time to java time for date-related parsing, formatting, and calculations. This guide is designed to help you determine if your cluster is impacted and, if so, prepare for the upgrade.
Convert date formats
To upgrade to Elasticsearch 8, you’ll need to convert any joda-time date formats to their java-time equivalents.
Impacted features
The switch to java time only impacts custom date and
date_nanos formats.
These formats are commonly used in:
If you don’t use custom date formats, you can skip the rest of this guide. Most custom date formats are compatible. However, several require an update.
To see if your date format is impacted, use the deprecation info API or the Kibana Upgrade Assistant.
Incompatible date formats
Custom date formats containing the following joda-time literals should be migrated.
-
Y(Year of era) -
Replace with
y.Example:
YYYY-MM-ddshould becomeyyyy-MM-dd.In java time,
Yis used for week-based year. UsingYin place ofycould result in off-by-one errors in year calculation.For pattern
YYYY-wwand date2019-01-01T00:00:00.000Zwill give2019-01For patternYYYY-wwand date2018-12-31T00:00:00.000Zwill give2019-01(counter-intuitive) because there is >4 days of that week in 2019 -
y(Year) -
Replace with
u.Example:
yyyy-MM-ddshould becomeuuuu-MM-dd.In java time,
yis used for year of era.ucan contain non-positive values whileycannot.ycan also be associated with an era field. -
C(Century of era) -
Century of era is not supported in java time. There is no replacement. Instead, we recommend you preprocess your input.
-
x(Week year) -
Replace with
Y.In java time,
xmeans zone-offset.Failure to properly convert
x(Week year) toYcould result in data loss. -
Z(Zone offset/id) -
Replace with multiple
X's.Zhas a similar meaning in java time. However, java time expects different numbers of literals to parse different forms.Consider migrating to
X, which gives you more control over how time is parsed. For example, the joda-time formatYYYY-MM-dd'T'hh:mm:ssZZaccepts the following dates:2010-01-01T01:02:03Z 2010-01-01T01:02:03+01 2010-01-01T01:02:03+01:02 2010-01-01T01:02:03+01:02:03
In java time, you cannot parse all these dates using a single format Instead, you must specify 3 separate formats:
2010-01-01T01:02:03Z 2010-01-01T01:02:03+01 both parsed with yyyy-MM-dd'T'hh:mm:ssX 2010-01-01T01:02:03+01:02 yyyy-MM-dd'T'hh:mm:ssXXX 2010-01-01T01:02:03+01:02:03 yyyy-MM-dd'T'hh:mm:ssXXXXX
The formats must then be delimited using
||:yyyy-MM-dd'T'hh:mm:ssX||yyyy-MM-dd'T'hh:mm:ssXXX||yyyy-MM-dd'T'hh:mm:ssXXXXX
The same applies if you expect your pattern to occur without a colon (
:): For example, theYYYY-MM-dd'T'hh:mm:ssZformat accepts the following date forms:2010-01-01T01:02:03Z 2010-01-01T01:02:03+01 2010-01-01T01:02:03+0102 2010-01-01T01:02:03+010203
To accept all these forms in java time, you must use the
||delimiter:yyyy-MM-dd'T'hh:mm:ssX||yyyy-MM-dd'T'hh:mm:ssXX||yyyy-MM-dd'T'hh:mm:ssXXXX
-
d(Day) -
In java time,
dis still interpreted as "day" but is less flexible.For example, the joda-time date format
YYYY-MM-ddaccepts2010-01-01or2010-01-1.In java time, you must use the
||delimiter to provide specify each format:yyyy-MM-dd||yyyy-MM-d
In java time,
dalso does not accept more than 2 digits. To accept days with more than two digits, you must include a text literal in your java-time date format. For example, to parse2010-01-00001, you must use the following java-time date format:yyyy-MM-'000'dd
-
e(Name of day) -
In java time,
eis still interpreted as "name of day" but does not parse short- or full-text forms.For example, the joda-time date format
EEE YYYY-MMaccepts bothWed 2020-01andWednesday 2020-01.To accept both of these dates in java time, you must specify each format using the
||delimiter:cccc yyyy-MM||ccc yyyy-MM
The joda-time literal
Eis interpreted as "day of week." The java-time literalcis interpreted as "localized day of week."Edoes not accept full-text day formats, such asWednesday. -
EEEEand similar text forms -
Support for full-text forms depends on the locale data provided with your Java Development Kit (JDK) and other implementation details. We recommend you test formats containing these patterns carefully before upgrading.
-
z(Time zone text) -
In java time,
zoutputs Z for Zulu when given a UTC timezone.
Test with your data
We strongly recommend you test any date format changes using real data before deploying in production.
Update index mappings
To update joda-time date formats in index mappings, you must create a new index with an updated mapping and reindex your data to it.
The following my-index-000001 index contains a mapping for the datetime field, a
date field with a custom joda-time date format.
GET my-index-000001/_mapping
{
"my-index-000001" : {
"mappings" : {
"properties" : {
"datetime": {
"type": "date",
"format": "yyyy/MM/dd HH:mm:ss||yyyy/MM/dd||epoch_millis"
}
}
}
}
}
To change the date format for the datetime field, create a separate index
containing an updated mapping and date format.
For example, the following my-index-000002 index changes the datetime field’s
date format to uuuu/MM/dd HH:mm:ss||uuuu/MM/dd||epoch_millis.
PUT my-index-000002
{
"mappings": {
"properties": {
"datetime": {
"type": "date",
"format": "uuuu/MM/dd HH:mm:ss||uuuu/MM/dd||epoch_millis"
}
}
}
}
Next, reindex data from the old index to the new index.
The following reindex API request reindexes data from
my-index-000001 to my-index-000002.
POST _reindex
{
"source": {
"index": "my-index-000001"
},
"dest": {
"index": "my-index-000002"
}
}
If you use index aliases, update them to point to the new index.
POST /_aliases
{
"actions" : [
{ "remove" : { "index" : "my-index-000001", "alias" : "my-index" } },
{ "add" : { "index" : "my-index-000002", "alias" : "my-index" } }
]
}